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| Praise for Artificial Intelligence in Education clearly written for easy understanding, this is an essential guide to 


this early moment in the fourth Industrial Revolution.” 


bi From International Organizations —Keith Krueger, CEO, Consortium for School Networking 
(COSN) 


Artiticial inteligence wil be dimptye but bos poopie “Artificial Intelligence in Education is a breakthrough that delves into 
understand that education is going to be in the first 


| ; a two very important and related subjects: moving to a more 
j frontline. This brilliantly reflective and forward-looking book ; : 

; H ee ne modern personalized curriculum and the role of AI in teaching 
helps the education community in navigating the storm, avoiding 
both the Scylla of fashionable denial of teaching knowledge and 


the Charybdis of romantic restoration of the old disciplinary 


and learning. The book provides an excellent overview of both of 
these areas, establishing a foundation to serve as a basis for 


: i bringing these two fields together towards improving education 
canon. Quite a daring intellectual undertaking!” 

—Dirk Vandamme, Deputy Director, Directorate for Education 

and Skills, Organization for Economic Cooperation and 

Development (OECD) 


for every student.” 
—Rob Abel, CEO, IMS Global Learning Consortium 


“Artificial Intelligence in Education is the first internationally 


“This is a must read for educators and all stakeholders comprehensive attempt to help policy makers and educators 


interested in the future of education which will be to read through the lines of artificial intelligence and find 


impacted—and more than likely transformed—by AI. The what is in there for them. Readers will certainly welcome the 


book is enjoyable and accessible as it models interdisciplinary analytical perspectives and, even more, the value propositions 


learning by weaving in examples from domains including that reaffirm the value of education in a world where many 
philosophy, science, engineering and popular culture. By staying spheres of daily life, from work to culture and social life, could be 
rooted in the science of learning, the authors provide a 


critical lens on both the potential benefits and risks of AI 


dramatically challenged by artificial intelligence.” 
—Francesc Pedró, chief, education policy, UNESCO 


without hyping the technology. Pll also keep it handy as a 


“Artificial Intelligence in Education is an important, if at times 
teady reference to the many ‘what’ and ‘how’ frameworks and 


disturbing, contribution to the debate on AI and provides a 


models that will help me to map a course forward though an detailed analysis on how it may affect the way teachers and 


ey F 
eee Ee niae re: students engage in education. The book describes how artificial 


—Jim Flanagan, chief operating and strategy officer, The 


te i 
International Society for Technology in Education (ISTE) E O pE oa ESEE 


individualization of learning, and on assessment, offering 
some tantalizing glimpses into the future (the end of exams, 


“A must-read resource that enables you to cut through the ; À 
your very own lifelong-learning companion) while not falling 


hype around AI in education and think deeply about 
designing the future of teaching and learning. Balanced and 


victim to tech-hype. The enormous ethical, technical and 


pedagogical challenges ahead are spelt out, and there is a real risk 


that the rapid advances in artificial intelligence products and 


services will outstrip education systems’ capacity to understand, 
manage and integrate them appropriately. As the authors 
conclude: “We can either leave it to others (the computer 
scientists, AI engineers and big tech companies) to decide how 
attificial intelligence in education unfolds, or we can engage in 
productive dialogue.” I commend this book to anyone 
concerned with the future of education in a digital world.” 


—Maic Durando, executive director, European Schoolnet 


From Corporations 


“The fourth Industrial Revolution will impact both K-12 
education and what we need to learn later in life in an 
unprecedented way. This is a comprehensive and elaborate 
synthesis of how AI will change what we need to learn, but 
also how we will learn it in the future.” 


—Ulrik Juul Christensen, chief executive officer—Area) Lyceum; 
executive chaitman—Area9 Group 


“To begin to realize the potential of AI in education, education 
leaders and stakeholders globally need a much deeper and shared 
understanding of how AI intersects with curriculum 
modernization as we shift towards competency-based learning 
models. This book provides the strongest foundation available 
for deepening that understanding. It is a must-read for anyone 
seeking to go beyond the hype of Al towards appropriate, 
precise, and empowering uses of these tools for learning.’ 


—Mati thy director of Worldwide Education Research, 
Maria Langworthy ee 


“The new book Artificial Intelligence in Education provides 
readers with both a view on what people should know to 
thrive in the era of AI, as well as how AI will impact the 
education industry and society more broadly. In addition to 
traditional knowledge and skills, learners in the future yU need 
more meta-learning skills and character building experiences to 


be successful. The education industry will use AI to help both 
learners and teachers be more successful in adapting to rapid 
change, and advancing AI systems will have better and better 
models of users' knowledge and goals. A recurring 
theme/warning in the book is that individuals must actively 
engage to shape these technological and economic forces 
towards desirable outcomes, or be prepared to be shaped by 


the forces in ways that are less desirable than they may 
wish.” 


—Jim Spohrer, director of Mapping AI Progress with Cognitive 
Opentech Group IBM 


“In these turbulent times with increasingly complex problems to 
solve, the need for better talent is essential for success, and 
survival. The role of AI as a means to leverage human 
intelligence is widely being explored. The application of AI to 
accelerating learning and making it more widely available is the 
very complex subject of this book. It is an extensive and 
comprehensive collection of frameworks and tutorials with 
appendices that make it usable by experts and novices alike. 


It is an essential tool for any leader or researcher exploring this 
field.” 


—John Abele, chaitman emeritus and co-founder Boston Scientific 


From Foundations and Non-Profit Organizations 


“In a world where information is readily available online, how can 
schools continue to be relevant? The emergence of artificial 
intelligence (AI) has exacerbated the need to have these 
conversations. Artificial Intelligence in Education immerses the reader 
in a discussion on what to teach students in the eta of AI and 
examines how AI is already demanding much needed updates to 
the school curriculum, including modernizing its content, 
focusing on core concepts, and embedding interdisciplinary 
themes and competencies with the end goal of making learning 


more enjoyable and useful in students’ lives. The second part of 
the book dives into the history of AI in education, its techniques 
and applications—including the way AI can help teachers be 
more effective, and finishes on a reflection about the social 
aspects of AI. This book is a must-read for educators and 
policy-makers who want to prepare schools to face the 
uncertainties of the future and keep them relevant.” 


—Amada Torres, Vice President, Studies, Insights, and Research, 
National Association of Independent Schools (NAIS) 


From A.I., EdTech, and Education Thought Leaders 


“This book provides a benchmark for understanding the 
impact of AI on the goals and methods of education in the 
21st century.” 

—Henty Kautz, founding director, Goergen Institute for Data 


Science; past-president, Association for the Advancement of 
Artificial Intelligence (AAAT) 


“AI is changing the knowledge and skills students need for 
success in a global, knowledge-based, innovation centered 
civilization. To accomplish these ambitious educational 
outcomes, AI is also enabling novel, powerful methods of 
teaching and learning. This valuable book also describes AI in 
education in the larger context of shifts in society.” 


—Chris Dede, Timothy E. Wirth Professor in Learning Technologies, 
Technology, Innovation, and Education Program, Graduate School of 
Education, Harvard University 


“Artificial Intelligence in Education is the best synthesis to date of the 
implications of code that learns—both the new aims it 
demands of secondary education and how educators can 
incorporate it into learning experiences.” 


—Tom Vander Ark, CEO ,Getting Smart, first director of education 
for the Bill & Melinda Gates Foundation 


“We are experiencing the fourth Industrial Revolution. We 
are increasingly living in a world of AI & robotics—a digitized 
and globalized world. As educators, particularly as educators, we 
need guidance to navigate our way in this complex and uncertain 
age of artificial intelligence. Artificial Intelligence in Education: 
Promises and Implications for Teaching and Learning is a guide that 
was worth waiting for.” 


—Anthony Mackay, CEO, National Center for Education and the 
Economy (NCEE) 


“The authors of Artificial Intelligence in Education brilliantly 
provide a ‘How to’ roadmap to harness the power of AI in 
education. This book should be required reading for every 
educator, policymaker, and curriculum designer.” 
— Robert Martellacci, EdTech Pioneer & co-founder & president, C21 


“Artificial Intelligence in Education is really two books in one: the 
first presents a comprehensive curriculum framework for 21st 
century learning; the second is a thorough survey of the uses of 
AI in learning. It is an invaluable resource for those 
concerned with the future of education.” 


—Tony Wagner, best-selling author of The Global Achievement Gap and 
Creating Innovators 
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Introduction: The Context 


Artificial intelligence (AT) is arguably žhe driving technological force of 
the first half of this century, and will transform virtually every industry, if 
not human endeavors at large.’ Businesses and governments worldwide 
ate pouring enormous sums of money into a very wide array of 
implementations, and dozens of start-ups are being funded to the tune of 
billions of dollars. 


2013 2044 O16 


Funding of Al startup companies worldwide, from 2013 to 2017 
(in millions of U.S. dollars). Source: Statista? 


2016 2047 


It would be naive to think that AI will not have an impact on 
education—au contraire, the possibilities there are profound yet, for the 
time being, overhyped as well. This book attempts to provide the right 
balance between reality and hype (per the Gartner diagram that follows), 
between true potential and wild extrapolations. Every new technology 
undergoes a period of intense growth of reputation and expectations, 
followed by a precipitous fall when it inevitably fails to live up to the 
expectations, after which there is a slower growth as the technology is 
developed and integrated into our lives. As visualized in the Gartner 


diagram, each technology can be said to reside somewhere on the curve 


1 Possibly matched only by biotechnology. 
2 https://www.statista.com/statistics/621 468/worldwide-artificial-intelligence-startup-company-funding-by- 


year 
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at any given time (for example Deep Learning, which is part of AI, is 
currently peaking). 


Biotech ~ Cultured oe 
Atfal Tisu 


Anot Aty 2008 


Innovation a Trough of y, oe 
gaer Expectations Gilluaiommont Be oF Savgitonm ent Productivity 


Time 
3 
Source: Gartner Inc. 


It is of course a risky proposition then, in a field moving so fast, to 
attempt to predict the future. As such, this work will likely be updated 
periodically to keep up with the developments (just as you would expect 
from softwate/apps). 

This book is organized around a somewhat glib quote: “There are 
only two problems in education: What we teach, and how we teach it.” 
Hence this book is divided into two parts, one focused on the What, and 
one on the How of AI in education. 


3 http://www.Gartner.com/SmarterWithGartner 
4 Dr Roger Schank, https://www.rogerschank.com/ 
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The What 
We're headed for a world where you're either going to be able to 


write algorithms ... or be replaced by algorithms. 
—Bridgewater hedge-fund billionaire Ray Dalio 


The first part of this book explores the question: What should students 
learn in an age of AI? And all the corollary, provocatively phrased 
questions: “If you can search, or have an intelligent agent find, anything, 


why learn anything? What is truly worth learning?” 


It is widely expected that AI will have an enormous impact on what 
we teach, as it will impact many occupations. Take for instance the 
Organization for Economic and Co-operative Development (OECD) 
Programme for the International Assessment of Adult Competencies 
(PIAAC)? survey, which measures adults’ proficiency in key information- 
processing skills—literacy, numeracy and problem solving in technology- 
rich environments—and gathers information and data on how adults use 
their skills at home and at work. Already, AI is matching more than 50% 


of adult human-proficiency levels, and closing in on another 36%. 


Proficiency Level OECD Adults Artificial Intelligence 


2 and below 53% Yes 
3 36% Close 
4-5 11% No 


Source: Elliott Stuart, “Computers and the Future of Skill Demand.”® 


Such progress is bound to continue at an accelerating, pace. IBM’s 
Open Leaderboard effort attempts to understand the progress being 
made by tracking many variables. According to IBM’s Leaderboard, AI 
should be getting into the realm of deeper self-learning by the early 2020s 


5 https://www.oecd.org/skills/piaac/ 
€ https://read.oecd-ilibrary.org/education/computers-and-the-future-of-skill-demand_9789264284395- 
en#pagel 
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and become capable of assisting, collaborating, coaching and mediating 
by the early 2030s. 


D 
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Learning from Labeled Training Data 
and Searching (Optimization 


Learn by Watching and Reading 
Education 


Learn by Doing and being Responsible 

Exploration) 

2015 2018 2021 2024 2027 2030 2033 2036 
Source: Jim Spohrer, IBM’ 


Given all the above, the What section makes a case for the necessity to 
focus on a broad, deep, and versatile education as a hedge against 
uncertain futures, which in turn means a reinvigorated focus on the deeper 
learning goals of a modern education: 


° Versatility, for robustness to face life and work. 
° Relevance, for applicability, and student motivation, 
° Transfer,‘ for broad future actionability. 


All of which are to be developed via: 


° Selective emphasis on important areas of traditional 


knowledge. 
° The addition of modern knowledge. 
° A focus on essential content and core concepts. 
° Interdisciplinarity, using real-world applications. 


° Embedded skills, character, and meta learning info the 
knowledge domains. 


7 IBM, 2017, Cognitive Opentech Group. 


* This refers to the transfer of knowledge from a domain it was learned in to another domain. 
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The How 


The second part of this book addresses the question: How can AI 
enhance and transform education? First, it is important to make the 
distinction between education technology (EdTech) at large and artificial 
intelligence in education (AIED) specifically. A quick summary of the 
affordances of EdTech is appropriate at this stage, as the taxonomy and 
ontology of the field is quite murky. Using the SAMR’ model that 
follows, the How section showcases how AIED will span all layers, with 


its maximum impact growing as it moves up the stack. 


Tech allows for the creation of vot 
Redes nition new tasks, previously «< au 


inconceivable 


= 
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š 
X Uraa Aurasma Bai Board) Book Creator 
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F odification task redesign ká To N 7 | 
k Edmodo Educreations iMovie Thinglink 
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uj : „Substitute, with functional _ . | 
= Augmentation improvement ay w h I 
L 
= 
+ S Tenoa 392 et fool 
stitution —s«_—i(“‘(‘é‘é we SHlutt@, with no 
a ubstitution funcional change. 


iBooks Calculator Dictionary.com Pages 


Substitution, augmentation, modification, and redefinition model (SAMR). 


Note that the examples shown in the preceding figure represent 
today’s apps, not tomorrow’s, and only serve to help explain the model. 
Often these apps are collapsed under one term, technology, and then 
there is much confusion about the potential of technology. This model 
helps us to delineate the different types of impact that technology can 
have, from mere substitution with no functional changes, all the way 
through the creation of new, previously inconceivable tasks as a result of 


technology. 


°’ Dr. Ruben Puentedura, http://www.hippasus.com/ 
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The Role of Assessments 


What gets measured gets managed —Lord Kelvin 


Assessments have been the hidden villain behind a lot of education 
debates, and a powerful one at enshrining institutional inertia. 
Repurposing the famous Aristotelian syllogism:'° 


Lack of, or poor, education is at the root of many human 
problems. 

Assessments define the education we get. 

Therefore, assessments are the root of many human 
problems. 


Although not a focus of this book, it is clear that assessments have an 
oversized role to play in the change process, and as part of the Al-driven 
systems of (mostly formative) assessments. 

Andreas Schleicher, ditector of the OECD’s Directorate of Education 
and Skills, publicly stated “What is easy to measure is also easy to 


oo 


automate,” thereby throwing the gauntlet to the assessment world to 


readjust its focus and thus drive change. 


Lastly 


Readers will have different priorities and interest in this topic. 
Policymakers and curriculum designers may initially favor the What 
section, while teachers and IT specialists may at first favor the How 
section. 

The What and How sections are therefore written to be independent 
of each other; the appendices also reflect an emphasis on digestibility, 
particularly for the technical details. 

Further, we are all pressed for time, so our writing philosophy is, to 
use Antoine de Saint-Exupéry’s words, “Perfection is attained not when 
there is no longer anything to add, but when there is no longer anything 
to take away.” This book is therefore not meant to be an in-depth 


10 “All men are mortal. Socrates is a man. Socrates is mortal.” https://en.wikipedia.org/wiki/Syllogism 
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academic piece, but rather it is meant to be concise and to the point, and 


adhere to Yuval Harari's philosophy: “In a world deluged with irrelevant | 
information, clarity is power.” ™ 
We wish you all very pleasant reading, and invite your feedback at: 


info@CurriculumRedesign.org 


" Harari, Y. (2018). 27 Lessons for the 21st Century. Spiegel & Grau. 
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Part 1 
What Should Students Learn? 
The Impact of AI on Curriculum 


Education is slow to change. Students are still hearing outdated 
justifications for curriculum choices that even the teachers find difficult 
to believe. While we are told that we won’t have a calculator with us 
every day, we all have extremely powerful calculators in our pockets. And 
not just calculators, but dictionaries, encyclopedias, books, papers, 
instructional videos, and platforms to ask questions and get answers. 
Access to powerful technology leads us to ask the question: if you can 
Google everything, why learn anything? Or, put less flippantly, what is 
truly worth learning? 


The Purposes of Education 


As with any discussion of value, context is highly important. To answer 
the question of what is worth learning, one must first ask about the 
purpose of the entire endeavor. 

This question is far from new, and the answers have changed over 
time. The purposes of education systems originally evolved around 
creating laborers, imparting religious knowledge, and the basic skills of 
literacy and numeracy. As the structure of society changed, education 
began to take on other practical, social, and emotional functions. 
Practically, school is now seen as a gateway to higher education and 
ultimately financial independence. In this view, education serves primarily 
as a seal of approval, signaling to future employers that prospective 
employees have met some minimum social standards of quality control. 

The social conception of educational systems has slowly matured as 
the needs of individuals in society, and society itself, have evolved; 
education is both a way to mold students to fulfill the needs of society, 
and a means by which students may become empowered to best fulfill 
their own needs. Finally, school is also perceived in emotional terms, as a 


place to be inspired and fall in love with learning (which, in a world 
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requiring ever-mote continuous adaptation, becomes an essential 
characteristic). 

Because learning is a lifelong endeavor, it is reasonable to consider the 
differences between the goals of education at the primary and secondary 
(K-12) levels, and the goals of learning later in life. Learning later in life 
tends to be necessary for three reasons: 


1. Economic: Specialization for careers, as the Opportunities of 
available jobs continue to shift. 

2. Civic: Staying informed about voting issues as the explosion of 
information continues to grow, and facts continue to become 
more difficult to pin down. 

3. Personal: For the personal pleasure of picking up new hobbies 


and continuing to grow and challenge oneself and relate to 
others. 


Primary and secondary school learning, by contrast, is specifically 
focused on developing the foundation for all future learning, both in 
terms of knowledge and competencies: 


1. Foundational knowledge: A solid foundation of knowledge on 
which to build when it comes time to learn more, or from 
which to apply what was learned in a real world setting. 

a. Core concepts: The most important concepts for 
students to understand in order to be able to make 
connections and meaning resulting in transfer." 

b. Essential content: The most important content 
knowledge that students must learn in order to 
internalize concepts and make informed decisions 
throughout their lives. 

2. Foundational competencies: The motivation and the ability to 
effectively activate knowledge when relevant and to learn 


more when it becomes necessary. 


Transfer: The process of making use of knowledge outside of the context in which it was learned. 
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a. Skills: What we do with what we know; how we use 
our creativity, critical thinking, communication, 
collaboration skills. 

b. Character: How we behave and engage in the world 
using our mindfulness, curiosity, courage, resilience, 
ethics, and leadership. 

c. Meta learning: How we reflect and adapt, i.e. with 


metacognition or a growth mindset. 


When we tun workshops about twenty-first century education, 
whether our audience is educators, school leaders, policy makers, or 
industry representatives, the answers we get are very similar when we ask, 
‘What will be important for students to learn to be prepared for the 
future?” 

Rarely does someone speak up with a particular book from English 
class or a particular period from History class. Nor do they speak up with 
a branch of mathematics or a topic from biology. Not one time has 
anyone said that it’s important for students to learn that mitochondria are 
the powerhouse of the cell. No, unequivocally the answers we receive 


over and over ate things like “how to think critically,’ “systems 


23) cç 23) cç 


thinking,” “ethics,” “communication,” “learning how to learn,” and so 
on. 

Intuitively, we know that content knowledge may be the least 
important thing that students retain from their schooling (and that they 
mostly don’t retain itl"), and yet time and again, well-intentioned 
educational efforts result in a never-ending bloat of material in the 
curriculum, with no time left to spend on that which is most important. 

It begins innocently enough. “Students need basic literacy and 
numeracy knowledge to learn higher level material.” But the issue is that 


it becomes very unclear how to draw the boundaries around what is 


13 Subirana, B., Bagiati, A., & Sarma, S. (2017). “On the forgetting of college academics: At ‘Ebbinghaus 
Speed”? Center for Brains, Minds, and Machines Memo (68): 1—12. 
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enough for the majority of people who are not going to become experts, 
and whether someone will go on to be an expert. 


Meanwhile, educators are teaching competencies such as 


collaboration, but often only as a byproduct, not in a way that is 
deliberate, systematic, demonstrable, and comprehensive. Because these 
competencies are harder to measure than content knowledge, 
assessments rarely focus on them, and it is that much more tempting to 
spend time on content, and hope that competencies are implicitly learned 
along the way. 

According to UNESCO: “Quality education systems have to enable 
learners to continuously adapt their competencies while continuously 
acquiring and even developing new ones. These competencies are diverse 
in scope; ranging from core skills, content knowledge, cognitive skills, 
soft skills, to occupational skills, they enable us to meet a complex 
demand or carry out a complex activity or task successfully or effectively 
in a certain context. Their typologies and approaches are as diverse as the 
entities—countties, organizations and individuals—that define them,”"* 

In Four-Dimensional Education” the Center for Cutticulum Redesign 
(CCR) synthesizes the curticula from 35 jurisdictions and organizations 
around the world, and together with input from teachers and 
administrators, as well as reports on expectations of employers, 
economists and futurists, creates a unifying framework that is: 


Comprehensive. There are no major elements missing. 
Compact. It is actionable and deployable. 
Uncorrelated. No duplication or confusion. 
Abstracted to the appropriate level. It is organized. 
Globally relevant. For broad acceptability. 


This framework breaks educational goals into four dimensions. 


1 UNESCO, http://www.unesco.org/new/en/education/themes/ strengthening-education-systems/quality- 
framework/desired-outcomes/ competencies 


15 Fadel, C., Bialik, M., and Trilling, B. (2015). Four-Dimensional Education: The Competencies Learners Need to 
Succeed. Center for Curriculum Redesign. 
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1. Knowledge—what we know and understand. 
2. Skills'*—-what we can do with what we know. 
3. Character'”'®—how we behave and engage in the world. 
4. Meta learning” —how we reflect and adapt. 


By looking across research, frameworks, and standards from around 
the world, we were able to create a list of 12 total competencies (in 
addition to knowledge) to represent the goals of a twenty-first century 
education. They are visualized through the following Venn diagram. 


"What we know and unt 
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Because the competencies (skills, character, and meta-learning) can be 

pretty complex, we conducted an in-depth analysis of the way they have 
p : 4 20 

been conceptualized, and broke them down into subcompetencies.” Our 


16 Bialik, M., & Fadel, C. (2015). “Skills for the 21st century: What should students learn?” Center for 
Curriculum Redesign. 

17 Alternatively called socio-emotional skills, non-cognitive skills, soft skills, etc. for more info see: 
http://curriculumredesign.org/wp-content/uploads/CCR-Decision-matrix-for-Character-terminology- 
FINAL.pdf 

18 Bialik, M., Bogan, M. Fadel, C., Horvathova, M. (2015) “Character Education for the 21st Century: What 
Should Students Learn?” Center for Curriculum Redesign. 

1 Bialik, M., & Fadel, C. (2015) “Meta-Learning for the 21st century: What should students learn?” Center for 
Curriculum Redesign. 


a https://curriculumredesign.org/framework-of-competencies-subcompetencies/ 
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current work involves translating these subcompetencies one step 
further: into classroom actions and habits. 

An extra challenge to this work is that realistically, there cannot be a 
class created to teach each one of these independently. In fact, the case 
could be made that they are actually dest taught in the context of 
knowledge! Therefore, we are now working on identifying the most 
conducive combinations and gathering ideas from experts in order to 
further expound on the competencies. 

This book, however, concentrates on the knowledge component. 
After all, it is the dimension that is the most directly and immediately 
affected by technological changes, and it deserves to be examined 
carefully in its own right.” 


Foundational Knowledge: What Do Students Need 
to Learn? 
Let us briefly consider an example. Although high school calculus is 
helpful for the 20-30% of bachelor’s degree students who enter college 
with a STEM major and are expected to take calculus,” what is the 
experience of the other 70-80%? Further, what is the experience of the 
nearly 30%” of high school graduates who do not enroll in college? What 
about the large number of students who do go to college and must take 
calculus as a prerequisite for their major, even if that career path does not 
include calculus? Currently, even in the best-case scenario, students are 
still spending the majority of their time on material they will never use 
again once they choose a specialization. The same thought experiment 
can be done with almost any subject matter. 

Knowledge taught in schools must be reorganized such that it is 
relevant to all students, while at the same time giving each student the 
Opportunities to study in depth the prerequisite knowledge they will need 


*! For the other three dimensions please visit http:/ /www.cutriculumredesign.org/our-work/ papers. 
Chen, X. (2013). STEM Attrition: College Students' Paths into and out of STEM Fields. Statistical Analysis Report. 
NCES 2014-001. National Center for Education Statistics. 

23 National Center for Education Statistics, https:// nces.ed.gov/fastfacts / display.asp?id=51 
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for whatever career trajectory they choose. This is a balance well worth 


striving for. 


Overview of Core Concepts 


What is learned in school ought to be useful to people well after they 
finish school. Approaching any new situation that requires using one’s 
knowledge, whether it is a real-world application or learning a more 
advanced topic in a given discipline, it involves leveraging what one has 
already learned. In either case, existing knowledge must be effectively 
used in a new context. The more robust a mastery one has developed of 
the fundamentals of a topic, the easier it is to leverage it to learn even 
more.” The question thus becomes, how can students’ understanding be 
developed in such a way that it is useful? 

In his book, Future Wise,” Harvard professor David Perkins makes the 
case that curriculum should work toward “expert amateurism” rather 
than try to instill expertise within subjects. While expertise privileges 
technical depth, expert amateurism aims for “a robust and flexible 
understanding of the fundamentals.” By internalizing the most important 
concepts of each discipline and across disciplines, which we will call core 
concepts, students are better equipped to deal with multifaceted problems 
and have a more diverse set of tools with which to interpret the world.” 

But the trap of the coverage mindset “in which students march 
through a textbook, page by page (or teachers through lecture notes) in a 
valiant attempt to traverse all the factual material within a prescribed 
time” is difficult to avoid.” Even frameworks that start with the 


important concepts typically break them down into minute topics that are 


24 Bower, G. H., & Hilgard, E. R. (1981) Theories of Learning, Eaglewood Cliffs, NJ; Prentice-Hall 

2 Perkins, D. (2014). Future Wise: Educating Our Children For A Changing World. John Wiley & Sons. 

26 One important consideration, then, is that the world is changing rapidly, and the knowledge that is 
necessary to interact with the world is thus changing accordingly (more on that later). 

27 Wiggins, G., & Mctighe, J. (2005). Understanding by Design, Expanded 2nd Ed. ASCD. 
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not taught in the context of the broader disciplinary or subject concepts, 
and assessments often cover material only at the most granular level.” 

While experts readily understand the connections between content 
details and higher-level concepts, novices do not automatically make 
those connections. In fact, seeing connections between pieces of 
information in a field is a defining feature of what it means to be an 
expert. 

If novices are only taught and assessed on detailed fragmentary 
knowledge, they may appear to understand the material but are unlikely 
to be able to make use of what they have learned. In order to transfer 
knowledge to a new setting or to use knowledge to build larger 
understandings that will be useful and transferable, content should be 
connected with concepts in a way that helps students to create meaning. 
(See appendix 1 for an example of tying topics to important concepts). 

One difficulty is that the most useful knowledge is that which experts 
apply without a second thought, the knowledge that defines their field 
and is usually left unsaid. This is what makes it difficult for experts to 
create a curriculum that is truly geared toward developing expert 
amateurism. They know how to expertly work with the content, but they 
cannot necessarily state the concepts that lead to that expertise 
explicitly.” Imagine being asked to explain how to balance while walking, 
or how to chew. This is one of the greatest and most overlooked 
difficulties in creating a K—12 curriculum that builds a foundation for 
learning in students. 


Overview of Essential Content 


If students are expected to build on the foundation of knowledge they 
develop in school, one important characteristic of this foundation is that 
it be sufficiently representative of all the ways students could choose to 
build on it. In other words, by exposing students to many fields of 


*8 Cooper, M. M, Posey, L. A, Underwood, S. M. (2017). “Core ideas and topics: Building up or drilling 
down?” Journal of Chemical Education. 


2 https://en.wikipedia.org/wiki/Curse_of_knowledge 
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human endeavor through pairing core concepts with essential content, 
schooling can allow children to survey the different possible life and 
career paths and make informed decisions as to which endeavors they 
would be best suited to. 

In the slightly less long-term view, content must be reconsidered on 
the basis of society’s new relationship with information. Storage of 
information was scatce for most of human history, but became abundant 
with the mass production of books. With personal computers, and finally 
the internet, there became an abundance of resources to manipulate 
information as well. Any simple piece of information can be found 
quickly online, and powerful computational tools are easily accessible. In 
addition, the evidence suggests that individuals forget academic content 
at the rate of 50% every two years,” and that what is known in a given 
field changes over time such that a predictable fraction of what is learned 
in school will be outdated by the time it would be useful in a professional 
setting.” In such a context, what is the essential content that is worth 
knowing, and not just searching for when it is needed, or learning if one 
chooses to specialize? 

Some content will be the medium that the concepts are taught 
through, and some limited, appropriate amount of content is worth 
internalizing to automaticity to build up more complex knowledge later, 
or to use in daily life. Content may serve as the way a concept is 
introduced because it is most exemplified in that context, or, it may 
demonstrate the generality of a concept in near or far. This will allow 
students to abstract and apply concepts to relevant situations they 
encounter in the future without being explicitly prompted (as authentic 
environments tend not to do). 

In order to maximize its relevance for a rapidly changing world and 
society, content needs to be modernized in two ways. First, key modern 
disciplines (engineering, wellness, sociology, etc.) that haven’t had time to 


be included into the curriculum must be added, and decisions must be 


30 Subirana, B., Bagiati, A., & Sarma, S, “On the forgetting of college academics,” 1-12. 
31 Arbesman, S. (2013). The Half-Life Of Facts: Why Everything We Know Has an Expiration Date. Penguin. 
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made about what parts of the current curriculum should be de- 
emphasized or eliminated. Second, the way in which (both traditional and 
modern) disciplines are taught should be modernized. For example, the 
pedagogical notion of a flipped classroom relies on the idea that 
technology can fundamentally change the structure of learning. As we 
will see in Part 2 of this book, there should be a push to move from mere 
technology acting as a mete substitution with no functional changes, to 
the creation of new, previously inconceivable tasks as a result of 
technology. 

Beyond these practical goals, it is generally deemed valuable that 
students have some capacity to appreciate the fields they are not actively 
pursuing, for civic engagement as well as for developing a multifaceted 
sense of personal meaning and connecting with others. 

This is a large set of goals, and in the process of making sure students 
ate exposed to the wide range of ideas and topics in a given discipline, 
classes are often accordingly designed to be sweeping surveys of the field. 
This in itself is not necessarily problematic, but presenting students with 
fragmented information (a common consequence of such a course 
design) makes it difficult for them to develop conceptual frameworks 
that they can later use to understand new information or to build upon 
existing information. 

The opposite of fragmented information is meaning. In his book, 
Realms of Meaning,” Philip Phenix makes the argument that making 
meaning is the essential human activity, and that education should help 
students learn the different ways that humanity has successfully 
developed to make different kinds of meanings. In this view, it may be 
said that students should get sufficient exposure to the different realms 
of making meaning.” 

In building a foundation for future learning and appreciation, meaning 


is a useful guiding principle, as it is closely related to feelings of 


* Phenix, P. H. (1964). Realms of Meaning a Philosophy of the Curriculum Jor General Education. McGraw-Hill. 
* Realms of meaning are related but independent of each other 
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purpose,” understanding,” and engagement.” This type of meaning- 
making and deep understanding of the logic of a field of study is exactly 
the type of thing that one cannot simply search for when needed, but 
rather, must know intuitively when and how to apply (in fact it is needed 
to know what to search for). It is also the type of thing that, unlike 
fragmented knowledge, is not easily forgotten, ot changed, with time. 


Making Meaning and the Impact of Algorithms 


Since ancient times, schools have been tasked with helping students build 
a foundational understanding on which they can later rely when they 
hone their expertise. Relevance, which determines how well students will 
be able to make meaning, has always been crucial for making sure that 
what is learned in school is transferrable. And yet, there is a particular 
urgency to the question now, especially as it relates to climate change, 
social disruptions, technological breakthroughs, and the changing 
landscape of employment opportunities. Although not the only 
important consideration (consider personal and civic functions described 
above), one of the most worrying implications of this question has to do 
with the changing landscape of occupations as algorithms become more 


prevalent. 

Employability 

One difficulty with designing curricula to expose students to what they 
will need to know for their careers is the rapid change in the landscape of 
available professions due to automation and offshoring; preparations we 


make to ready students for the world of today will be outdated by the 


time they graduate. However, automation is not affecting all types of jobs 


3 Frankl, V. E. (1985). Man's Search For Meaning. Simon and Schuster. 

35 Understanding is in a very real sense equivalent to making meaning, since it must always be active on the 
part of the learner. ek 

36 Shernoff, D. J., Csikszentmihalyi, M., Schneider, B., & Shernoff, E. S. (2003). “Student engagement in high 
school classrooms from the perspective of flow theory.” School Psychology Quarterly,18 (2): 158-176. 
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equally. So far, the types of jobs that can be automated are ones that 
involve routine tasks.” 
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Changing prevalence of types of tasks required for work over time. 
Source: Autor and Price 


Tasks that are routine are easily automatable, since a computer 
program can learn to execute a series of steps and follow rules (an 
algorithm). This is evident in the changing proportions of types of jobs in 
the figure below; the two types of jobs that have increased have been 
non-routine interpersonal (such as a consultant) and non-routine analytic 
(such as an engineer), Routine manual (such as factory jobs) have 
decreased, as have non-toutine cognitive (such as filing paperwork). 
Non-toutine manual jobs (such as plumbing) did decrease but then seem 
to have hit a plateau, as there continues to be a baseline need for them. 

Several organizations (Oxford University,* OECD,” PwC,” 
McKinsey,“ and others) have tried to quantify the impact of automation 


37 Autor, D. and Price, B. (2013). “The changing task composition of the US labor market: An update of 
Autor, Levy, and Murnane.” MIT Mimeograph. 


38 Frey and Osborne. (2013). The Future of Employment: How Susceptible are Jobs to Computerization? University of 
Oxford. 

9 Arntz, YE T. Gregory and U. Zierahn. (2016). The Risk of Automation for Jobs in OECD Countries: A 
Comparative Analysis. OECD Social, Employment and Migration Working Papers, No. 189, OECD Publishing. 
“ Berriman, R. and Hawksworth, J. (2017). Will Robots Steal Our Jobs? The Potential Impact Of Automation On The 
UK And Other Major Economies. UK Economic Outlook. https://www.pwc.co.uk/economic- 

services /ukeo/pwcukeo-section-4-automation-march-2017-v2.pdf 


#1 McKinsey Global. (2017).“Automation and the future of work—briefing note prepared for Necker Island 
meeting on education.” 
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on occupations, with a growing concern about jobsolescence. ‘Their 
numbers range from 9% (OECD) to about 50% (Oxford University). 
The subject has also recently gained the public’s attention, with articles 
ranging from a doomsday scenario to a cheerful utopia, and those trying 
to describe various nuanced positions in between.” 

The progress of AI has been nothing short of stunning to a large 
segment of even the technologically literate.“*“* One way to examine this 
trend is through the lens of Bloom’s taxonomies, extended to the 
cognitive,” affective,“ and psychomotor“ domains. These were made to 
understand and categorize increasing complexity in thought, emotion, 
and movement, respectively. The following figure shows that existing 
algorithms already encroach on significant portions of human 


capabilities, and that is only the beginning of the impact. 


#2 See: Brooks, Rodney. (2017) “The seven deadly sins of AI predictions.” MIT Technology Review. See also: 
Chui, Michael, Manyika, James, and Miremadi, Mehdi. (2015) “Four Fundamentals of Workplace 
Automation.” McKinsey Quarterly. http: //www.mckinsey.com/business-functions /digital-mckinsey/our- 
insights /four-fundamentals-of-workplace-automation 

Hensel, A. (2017) “How robots will really take our jobs.” VentureBeat. 

Jones, M. (2017) “Yes, the robots will steal our jobs. And that’s fine.” The Washington Post. 

Shewan, D. (2017) “Robots will destroy our jobs—and we're not ready for it.” The Guardian: Technology 
Surowiecki, J. (2017) “Robopocalypse Not” The Great Tech Panic of 2017. Wired. 

43 It is estimated that historically the exponential acceleration of technology is due to three main factors: about 
66% due to hardwate speed, about 20% due to solid data sets, and about 10% to the algorithms themselves. 
But now, advances in the fundamental algorithms for learning are becoming the main driver of progress. 

44 Anthes, G. (2017) “Artificial intelligence poised to ride a new wave.” Communications of the ACM 60 (7): 19- 
21. https://cacm.acm.org/magazines/2017/7/21 8862-artificial-intelligence-poised-to-ride-a-new- 

wave/ fulltext 

43 Krathwohl, D. R. (2002). “A revision of Bloom’s taxonomy: An overview.” Theory Into Practice 41 (4): 212- 
218. 

46 Krathwohl, D. R., Bloom, B. S., & Masia, B. B. (1964). Taxonomy of Educational Objectives, Handbook I: Affective 
Domain. David McKay Co. 

47 Simpson, E. (1971). “Educational objectives in the psychomotor domain. Behavioral objectives in 
curriculum development: Selected readings and bibliography.” 60 (2). 

https: //files.eric.ed.gov/fulltext/ED010368.pdf 

See also Hill, K., Fadel, C., and Bialik, M. (2018). Psychomotor Skills For The 21st Century: What Should Students 
Learn? Center for Curriculum Redesign. hitps://curriculumredesign.org/wp-content/uploads/Psychomotor- 
Skills-CCR-May2018.pdf 
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Augmented Intelligence 


How clean is this divide between human tasks and computer tasks? 
Once a task has been automated, is there any role left for humans? 
Chess is a good example to consider because it is a game that one 
could imagine playing in a human, holistic way using intuition, or in a 
robotic, algorithmic way using powerful calculation. When in 1997 the 
computer Deep Blue beat the world champion, Garry Kasparov, at 
chess, it seemed to add chess to the list of activities in which humans 
have been outpaced by computers. Similarly, the game of Go has been 
conquered in recent times, to the point of algorithms even using 
innovative strategies that have not been used by human players.’ 

However, while it is true that computers can now beat humans at 
chess, combinations of the two—computers and humans working 
together—appear to be more effective than either alone. The results of 
freestyle chess competitions have shown that amateur chess players using 
computers can beat computers alone, grandmasters alone, and even 
grandmasters using a weaker computer.” This is a great example of a 
situation which appears on the surface to be yet another case of 
computers encroaching on what was once deemed a uniquely human 
challenge, but turns out to be an opportunity for humans to use 
algorithms as tools for doing what humans do best, even better. This idea 
has been referred to as “augmented intelligence” and it is the key to 
understanding the role of humans with regard to computers, and thus 
holds implications for the goals of an education. 


+8 https://en.wikipedia.org/wiki/AlphaGo 
49 Brynjolfsson, E., & McAfee, A. (2014). The Second Machine Age: Work, Progress, and Prosperity in a Time of 
Brilliant Technologies. WW Norton & Company. 
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Comparison of computers humans and human-computer teams. 
Source: Adapted from PARC.” 


A similar process applies to many professional shifts. Calculators 
didn’t replace mathematicians, but rather boosted their abilities. Word 
processors did not replace writers, but rather gave them more power as 
they wrote and edited their work. Even though the changes taking place 
due to AI are likely more transformative than those shifts, that does not 
mean that AI will not also be used best as a tool, if the next generation is 
effectively trained to make best use of it. 

So where are machines best suited, and where could humans expect 
an enduring role, and leverage the power of machines? Our summary 
based on the current state of affairs is as follows: 


Areas Where Machines Best Humans 


e Repetitive/predictive tasks. 
Tasks that hinge on computational power. 
° Classifying huge amounts of data and inputs. 


e Making decisions based on concrete rules. 


° Kefik, M. (2017). “Half-human, half-computer? Meet the modern centaur.” PARC Blog. 
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Areas Where Humans Best Machines 


e Experiencing authentic emotions and building relationships. 

e Formulating questions and explanations across scales and 
soutces. 

e Deciding how to use limited resources across dimensions 
strategically (including which tasks machines should be doing 
and what data to give them).” 

e Making products and results usable for humans and 


communicating about them. 


Making decisions according to abstract values. 


If all tasks were perfectly algorithmic and able to be subdivided, 
algorithms could handle it all. But most of the time, humans are required 
to frame the problem, choose the data, decide how the pieces fit 
together, communicate its value to others, make judgments according to 
values, and so on. Even AI, while it may be free of the constraint of 
algorithms, will still need to be designed, trained, and situated within a 
larger process. Although many parts of jobs are likely be automatable, 
there are still important roles for humans to play, if they are properly 


prepared. 


31 Additionally, humans have biases, and their algorithms and data sets can reflect them, so it will be important 


to account for unintended consequences of the way information is structured and combined. 
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Implications for Education and What Students Need to Know 


Considering the exponential advances of AI and its disruptive nature on 
occupations and tasks, as well as all the other factors of social and 
personal instabilities, what is a wise strategy for humans to adopt, when it 
comes to education?” 

Logic would indicate that, in times of unpredictable change, 
adaptability and resourcefulness would be essential. This in turn makes a 
case for a mote versatile education, where one could be broadly trained in 
a number of domains, and learn the skills and character qualities needed 
to thrive and adapt to other endeavors. This is in a sense what education 
has always tried to achieve—a solid foundation that translates to 
pteparedness for future challenges—but this has to be done mote 


effectively than ever, given the magnitude of the disruptions. 
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T-shaped and M-shaped person, for comparison. 
Source: CCR based on Spohrer. 


IBM’s Jim Spohrer coined the term T-shaped person” to indicate the 
cognitive profile of someone whose knowledge is both broad and deep, 
not one or the other. To include the projected changes in the workforce, 
we expand this model by describing an M-shaped person, who develops 
several depths over their lifetime. 


52 s 
Certainly education cannot solve everything—there are political and legislative discussions that must be 
had—but our focus here is on education. 


5 https://www.slideshare.net/spohrer/ t-shaped-people-20130628-v5 
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Two ways of reaching the goal of K-12 education: the traditional method, 
which assumes that transfer can only be a result of expertise (curved line) 
and the proposed method (wavy line) that alternates between gaining 
transfer and expertise. Source: CCR. 


Education is always about transfer (the process of making use of 
knowledge outside of the context in which it was learned) and expertise 
(a highly developed understanding of some domain of knowledge, 
including particular ways of perceiving and interpreting information).”* 
But now, more than ever, there is the need to reconsider the relationship 
between expertise and transfer and to make them the focus of an 
education” in a deliberate, systematic, comprehensive, and demonstrable 
way.” 

The diagram that follows shows how increasing technological 
processes provide the opportunity for the emphasis of curriculum to be 
flipped” so students spend more time focused on transfer and expertise 
via concepts rather than on learning content that can now be easily 
accessed and manipulated. This is similar to the way that flipped 
classroom pedagogy incorporates technology for learning content so that 


34 For example, see: Simonton, D. K. (2000). “Creative development as acquired expertise: Theoretical issues 
and an empirical test.” Developmental Review 20 (2): 283-318. 

35 Expertise was always understood to be important to focus on, but in the ideal balance, both expertise and 
transfer would be emphasized. 

36 One of CCR’s key mantras: Yes, here and there, now and then, to a certain extent, such efforts take place. 
CCR’s goal is to make them considerably more “deliberate, systematic, comprehensive, and demonstrable.” 

57 Here we are adapting the language of flipped classroom but actually referring to a flip in the value placed on 


different priorities, rather than on classroom strategies. 
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class time can be used for activity-based concept learning, but with a 
focus on “the what” rather than on “the how.” 

This is directly related to technological changes; search and AI 
algorithms first encroached on the more fundamental aspects of 
knowledge (basic facts and processes), but are now reaching the level of 
specific expertise. Algorithms are expanding upward in their abilities to 
provide meaning, which can be seen as vety beneficial rather than 
threatening. Rather than spend undue amounts of classroom time 
amassing information, the time can now be much better spent reaching 
the more profound goals of an education, namely expertise and transfer, 


preparing students for the types of tasks they will actually be expected to 
do in the workforce.” 


Flipping the Curriculum 


Center for Curriculum Redesign 


Flipping the curriculum to account for the encroachment of search and Al. 
Source: CCR 


Considering the changes taking place, we are reminded that content 
must be modernized as well. One way to approach this is to remove 
portions of the curriculum that are outdated. A useful metaphor might be 
one of the game Jenga,” where players remove the non-load-bearing 
blocks without weakening the edifice, thereby keeping the essential 
structure. (Of course, it is possible to remove too much; the goal is to 


*8 https://en.wikipedia.org/wiki/Flipped_classroom 
5 This is important but, as a reminder, it is not the only aim. 
© https://en.wikipedia.org/wiki/Jenga 
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keep what is essential). Additionally, valuable contemporary topics should 
be added to the classic content to update its relevance. Finally, the way 
content is taught should be modernized, making use of new educational 
as well as professional tools—although using old tools is still preferable 
to using new tools mindlessly. 

Just as content can overpower concepts, expertise can become the 
focus of education, at the expense of transfer. Consider an example from 
mathematics: drilling procedures for solving trigonometric equations 
involving arc secants by hand was useful for training land surveyors in a 
non-digital world, or might have been useful for those who went on to 
study engineering, just as memorizing the Krebs cycle was once useful 
for those who went on to a career in biology. But for the majority of 
modern students, both examples often end up serving no real purpose 
and, in fact, can be easily accomplished by using computers for search or 
calculation. 

This approach, which focuses on expertise and eventually leads to 
transfer, is not the only way to reach the goal of combining transfer and 
expertise, and it has significant drawbacks. Namely, if one stops the 
development of knowledge within a subject early (as many do, and in 
most cases K-12 encourages by design), then the potential for transfer 
remains incredibly low. The detailed understanding remains limited to the 
domain in which it was learned, and is not useful. It is possible, as the 
preceding transfer/expertise figure illustrated, to alternate between 
teaching for transfer (breadth) and teaching for expertise (depth), such 
that if one stops early, they’ve still gained important and transferrable 
understandings.” i 

For example, it is possible to gain an understanding of the scientific 
way of viewing the world by earning a PhD in a scientific field, 
painstakingly developing expertise by learning the relevant background 
knowledge, reading dense primary research articles, and designing and 
executing research studies. But shouldn’t the ability to consider the 


61 It may be the case that different disciplines have slightly different curves. 
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testability and uncertainty of claims—something inherent to science—be 
internalized sooner rather than later? Shouldn’t it be conveyed in every 
relevant example of scientific knowledge that is taught? Most 
practitioners of a field will claim that no one piece of information is as 
important as learning to think in a particular way. Additionally, this 
“thinking in a particular way” is something that remains inaccessible to 
computers. Certainly, we are now bombarded with information we are 
expected to be able to evaluate, for which we need to have learned the 
general frameworks that would allow us to ask the right questions. 
Education needs to be restructured to reflect that. 


The Importance of Meaning 


What makes a foundation of knowledge solid? What makes it fragile? The 
key lies in making meaning. 

A useful understanding is one that is meaningful: ideas are connected 
according to their relationships and applicability. In some sense, making 
meaning and understanding can be said to be synonymous. Experts, who 
have studied their field in great depth and have devoted their lives to a 
particular way of making meaning, develop an understanding so deep 
that they are often able to use their intuition to understand how to 
approach new challenges in their field. Of course, once one has learned a 
particular concept in great depth from many angles, it is natural that they 
will have developed an intuition for it. 

But can intuition be developed in those who will not go on to 
specialize in a subject? Can students be educated to recognize situations 
to which their knowledge may be applicable, and have informed 
strategies for how to approach the unknown ftom relevant perspectives, 
without developing full technical expertise in all subjects?” We believe 
that they can, if knowledge is presented in ways that facilitate making 
meaning, and that which is most fundamental is made explicit. 


8 Which is, of course, not possible. 
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Intuition 


Suppose you are a local in a particular city, and you have walked the 
streets your entire life and know them without having to think about it. 
No matter where you find yourself, you understand the general area you 
ate in and how it relates to other areas you have visited. You might be an 
expert in the layout of that city. 

Now, suppose you are trying to teach someone who is new to the city 
how to get around. You could provide them with a map and start at the 
top and have them memorize it square by square or pixel by pixel. After 
all, once you’ve finished this exercise, they would presumably have a very 
deep, detailed understanding of the city. Or you can just tell them the 
exact directions and local landmarks to get from one place to another 
every time they need to go anywhere, and if they stay in the city long 
enough they will probably put together a reasonable mental map. 

Alternatively, you could simply convey to them the largest scale of 
organization and the useful landmarks of the city. For example, you 
could tell them the river divides the town into north and south, there is a 
main street with neighborhoods that branch off of it, and there is a bus 
that loops along the perimeter of the city. This method provides a 
foundation of knowledge that is more meaningful and able to be 
expanded as the stranger learns more about the city, whether they plan 
on moving to the city or are there as a tourist. The description that 
provides the most essential components of the city is more useful, as it 
helps the stranger to develop an intuition for how to approach making 
meaning from the layout and landmarks of this city. If they find 
themselves in an unknown neighborhood, they can look for the river, 
and find their way, adding what they learn from the experience to their 
understanding of the city. 

That is truly the test of a foundation: can it be built upon, or is it 
going to collapse under the pressure of new material? If all the pieces 
have been added in a way that creates meaningful relationships with all 
the existing pieces, then it is clear where new pieces ought to be 
connected. If all the pieces are being collected together in the hope of an 
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emergent meaning arising in the future, any addition to this collection 
will just be another fragment of an idea to remember, and could be easily 
lost or confused. 

It may seem like some topics are so technical that there is no way to 
preserve meaning in the majority of the pieces as they are added, or to 
develop an intuition without developing comprehensive expertise; 
experts built their understanding in the current system, after all, and 
therefore may often default to believing that their own journey should be 
the journey of anyone learning their discipline. 

It may be, however, that an expert is in the worst position to decide 
how to introduce students to their discipline. It is common for them to 
succumb to the “curse of knowledge,” by which they are unable see 
how novices perceive material, and thus how it should best be presented. 
Someone with a deep passion for the material and to whom it came easily 
may have particular trouble imagining the experience of someone who 


Just wants a handle on the foundation, ot a struggling learner. 
Relevance: Mobilizing Knowledge 


Research shows that information is filtered through our perception based 
on our frameworks for understanding it and our goals. This applies to the 
lowest levels of perception™ as well as to the higher levels of cognition. 
If students’ brains cannot find the usefulness of information, it is more 
likely to be difficult to integrate (understand) the information in a 
meaningful way. For knowledge to be relevant, it does not necessarily 
have to be useful in a concrete way; it may be useful for solving an 
abstract problem or understanding a confusing idea, 

Furthermore, relevance has a direct relationship with student 


motivation. The information gap theory of curiosity posits that a 
De 


& Wieman, C. (2007). “The ‘curse of knowledge,’ or why intuition about teaching often fails.” APS News 16 
(10). 

> For example: Gauthier, I., Skudlarski, P., Gore, J. C. & Anderson, A.W. (2000). “Expertise for cars and 
birds recruits brain areas involved in face recognition.” Nature Neuroscience 3 (2):191-197. 

6 For example: Mack, A. and Rock, I. (1998). Inattentional Blindness. MIT Press, 


& Loewenstein, G. (1 994). “The Psychology of curiosity: A review and reinterpretation.” Psychological Bulletin 
116 (1), 75-98. 
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powerful form of motivation arises when there is awareness of a gap in 
one’s understanding. The gap, however, must be of manageable size; if it 
is too big or too small, students will be uninterested or intimidated, 
respectively.’ Additionally, people like to seek out information on topics 
they enjoy thinking about, and avoid information on topics they don’t 
enjoy thinking about. This last phenomenon, the ostrich effect, is 
particularly interesting, as it shows that relevance is highly subjective, 
taking into account emotional valence that may sway the way someone 
approaches potentially relevant but unpleasant information.” 

David Perkins says “Knowledge is like a bicycle. That is, knowledge is 
for going somewhere. If we know something about the French 
Revolution or the nature of democracy or Bayesian probability or 
Opportunity cost, we want to go somewhere with it. Maybe we want to 
understand an issue in the headlines or think about a medical decision or 
get a project off the ground in the most effective way. For any of these 
missions and thousands more, we want to go somewhere with what we 
know.”” Even going on a leisurely bike tide for pleasure requires the 
bicycle to be functioning. Put another way, knowledge should not be in a 
void, tather it should be useful for something. Acquiring this kind of 
knowledge is critical for complementing, rather than competing with, 
computers, which are not able to spontaneously see connections and 
transfer knowledge to new contexts; humans must do this formulation 
work by defining the problem, and then recruit computers to do the 
work within each of the problems that requires computational power. 

The idea of functional knowledge is often connected to vatious 
pedagogies that focus on active learning.” As John Dewey said, “only in 
education, never in the life of a farmer, sailor, merchant, physician, or 


€ McClelland, D. C., Atkinson, J. W., Clark, P. W., and Lowell, E. L. (1953) The Achievement Motive. Appleton- 
Century—Crofts. 

6 https://en.wikipedia.org/wiki/Ostrich_effect 

© Golman, R.and Loewenstein, G. (2015). “Curiosity, information gaps, and the utility of knowledge.” 
Available at SSRN: https://ssrn.com/abstract=2149362 or http://dx.doi.org/10.2139/ssrn.2149362 

7 Perkins, D. Future Wise. 

” Including self-directed learning, experiential learning, expeditionary learning, learning-by-doing, inquiry 


learning, hands-on learning, project based learning, problem based learning, discovery learning, and so on. 
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laboratory experimenter, does knowledge mean primarily a store of 
information aloof from doing.”” This was reinforced by the seminal 
work by Benjamin Bloom,” discussed above; in this framework, higher 
levels of cognitive complexity require greater mobilization of knowledge 
on the part of the learner, from remembering, to understanding, to 
applying, to analyzing, to evaluating, to creating.” Thus, many people 
have worked to solve the problem of imparting usable knowledge 
through pedagogical approaches that focus on the students’ active 
construction and application of knowledge. These approaches shift the 
focus from information transfer to student learning, and from “knowing” 
to “doing.” After all, learning knowledge in a context of its application 
ensures that it has at least that meaning for the student to use in their 
conceptual organization. As we will see, however, this is just part of the 
picture. 


Developing a Toolbox of Concepts 


Any particular piece of knowledge could in theory be useful in a variety 
of situations, but it is ultimately up to the student to know how and when 
to use it, and this is key to the type of understanding that ought to be the 
aim of education. For example, what use is the knowledge of the 
definition of confirmation bias if a student never checks whether they are 
succumbing to it? This fits with the common way of describing learning 
as developing a toolbox, meaning each student is honing their proficiency 
with a set of tools, which are in this case concepts they have learned to 
use appropriately, 

There are some concepts, however, that are more powerful than 
others. The concept of brute force versus elegance in mathematics, for 


example, crystallizes an important idea about the possible approaches to 
a eee 


” Dewey, J. (1916). Democracy and Education. https://www.gutenberg.org/files/852/852-h/852-h.htm 

73 Bloom, B. S., Engelhart, M. D., Hill, H. H., Furst, E. J., & Krathwhol, D. R. (1 956). Taxonomy of Educational 

Paini The Classification of Educational Goals, Handbook I: Cognitive Domain. David McKay Company. Inc, New 
ork. 

74 Anderson, L. W., Krathwohl, D. R., and others, (2001). A Taxonomy For Learning, Teaching, and Assessing: A 

Revision Of Bloom’s Taxonomy Of Educational Objectives, Pearson; and Krathwohl, D. R. (2002). “A revision of 

bloom’s taxonomy: An overview. Theory Into Practice 41 (4): 212-218. 
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solving a problem; being aware of the type of strategy one is employing 
(by using this concept as a tool) will help all students, regardless of 
whether or not they specialize in a science, technology, engineering or 
math (STEM) field. For this reason, this can be thought of as a power 
tool of thought. Although it is learned in a particular context, it has a 
disproportionate amount of utility for all students. 

Core concepts are the power tools of knowledge and should be of 
high priority in the curriculum, compared to concepts that offer less 
leverage for future learning, or worse, fragmentary knowledge that is 
inappropriately tied to the context it was learned in, and thus difficult to 


use as a tool in new contexts. 


Transfer: Using Learned Knowledge in New Circumstances 


Another way to frame this challenge is as transfer (the process of using a 
conceptual tool outside of the context in which it was learned). A lot of 
research” has focused on the puzzling findings around the cases in which 
transfer does and doesn’t occur, but recently, research has suggested that 
a more productive way to conceptualize transfer is that it is almost always 
occurring, but it is not always being done in the ways that teachers hope 
for. 

In essence, transfer can be conceptualized simply as activating a set of 
mental resources to understand some new information.” It is a natural 
process of learning, in which one leverages what they already understand 
to figure out something they do not yet understand” 7? If students are 
applying the tools incorrectly or failing to apply them when they are 
appropriate, that is an indication that the meaning that they’ve made is 


not complete or accurate in some way. Rather than “failing to transfer,” 


735 For example: Barnett, S. M., & Ceci, S. J. (2002). “When and where do we apply what we learn? A 
taxonomy for far transfer.” Psychological Bulletin 128 (4): 612-637. 

% Bransford, J. D. & Schwartz, D. L. (1999). “Rethinking transfer: A simple proposal with multiple 
implications.” Review of Research in Education 24: 61—100. 

7 Billett, S. (2013). “Recasting transfer as a socio-personal process of adaptable learning.” Educational Research 
Review 8: 5-13. 

78 Wiser, M., Smith, C. L., Doubler, S., & Asbell-Clarke, J. (2009). Learning Progressions as a Tool for Curriculum. 
Paper presented at the Learning Progressions in Science (LeaPS) Conference, June 2009, Iowa City, IA. 
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these students are simply transferring a tool in a way or to a context that 
does not apply. It also happens to be exactly what computers cannot do: 
figure out how to do something they have not been trained on without 
learning it from scratch. 

How does transfer take place? When confronted with some new 
problem, situation, or information, the first thing our brains do is try to 
find a pattern that matches a pattern they have already learned. They 
might find an abstract pattern for which they have tools, such as word 
problem, division, or poems, and activate appropriate tools to that 
category. This is called high road transfer. By contrast, low road transfer 
is when our brains notice a pattern match between the superficial 
features of the new information, and some superficial features of 
previous experiences.” 

For example, suppose a student is asked to solve this word problem: 
“Four children have 16 blocks. Their teacher asks them to share the 
blocks equally between them. How many blocks will each child receive?” 
To solve the problem, the student might use high-road transfer and 
realize that to share the blocks equally, they must use division. Or, if the 
student uses low-road transfer, they might notice that the problem 
structure or vocabulary are similar to other problems that they have 
solved in the past and know to follow the same procedure to solve the 
problem. Assuming the students have seen a similarly worded problem in 
the past, both types of transfer would successfully get the correct answer 
(4 blocks). However, high-road transfer allows the student to organize 
expetiences based on their deeper meaning, and thus will ultimately be 
more useful for transferring knowledge learned in school to questions 
and experiences after school. 

In terms of potential for transfer, core concepts, as described in the 
previous section, can have a high return on investment, in that they are 
applicable to many contexts, without needing to be learned in each one 
separately. If the goal of the curriculum is to allow students to construct 


7 Sometimes this happens cleanly, but often a particular situation will somewhat match several patterns. This 


accounts for students’ inconsistent application of certain concepts that are so puzzling to teachers. 
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transferrable knowledge, then core concepts should be central to the 
construction of the curriculum. In fact, one way of conceptualizing core 
concepts is that they are what allow transfer to take place. 

It is tempting to think that all learning could be as painless as learning 
one’s native language. This process happens naturally as children engage 
with the world, and the result is a truly impressive, complex knowledge 
base. However, learning another language in school does not usually 
include the option of sending students to a foreign country, and subjects 
such as math don’t have an equivalent construct of math country that 
students could visit even if they wanted to. Learning a foreign language in 
school involves the explicit discussion of grammar rules, declensions, 
conjugations, and so on. This is because hoping that one absorbs an 
entire language without being shown the lines along which transfer can 
take place is clearly inefficient. In a class of Spanish as a foreign language, 
a core concept may be situated around “verbs that end with -ar”. This 
way, when a student studies -ar verbs they are not only learning those 
words, they are learning how to treat certain words they have not yet 
encountered, and how to spot those situations in which their knowledge 
is transferrable. 


Realms of Meaning 


In the 1964 book Realms of Meaning,” Philip Phenix makes the argument 
that curriculum design should prioritize Opportunities for students to 
make meaning, and that disciplines should thus be grouped by the ways 
in which they make meaning: their typical methods, learning ideas, and 
characteristic structures. For example, math and linguistics both make 
meaning using systems of symbols and agreed upon rules. This way of 
making meaning has been a successful approach and it is worthwhile for 
students to understand what it means to make meaning in that particular 
way. Newer disciplines can be sorted into the same categories (for 


example, computer science would belong to the symbolic realm). 


= Phenix, P. H. (1964). Realms of Meaning a Philosophy of the Curriculum for General Education. McGraw-Hill. 
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This is the most abstract level of meaning, in that it is an entire 
approach to making meaning. Someone who is a scientist has internalized 
the empiric way of making meaning, and will likely look for evidence and 
consider alternate explanations even in matters of their personal lives. It 
is along these lines that interdisciplinary work is often most difficult; 
when values and methods of inquiry do not align it is hard to find a 
starting point or see each other’s position through their eyes. If a K-12 
education is to provide a foundation, that foundation should include an 
intuition for the different possible ways of making meaning. The Realms of 


Meaning are: 


Symbolic: Systems of symbolic structures with socially 
accepted rules of formation and transformation (for 
example, math, linguistics, computer science, etc.). 
Empiric: Probable empirical truths framed in accordance 
with certain rules of evidence and verification, and using 
certain systems of analytical abstraction (for example, 
physics, biology, etc.). 

Aesthetic: Patterns of the inner lives of humans (for 
example,Visual arts, Musical Arts, Arts of Movement, and 
Literature). 

Personal:®! Knowledge of self and others, learned through 
experience (for example, psychology, philosophy, 
literature, religion, in their existential aspects). 

Ethical: Personal conduct that is based on free, 
responsible, deliberate decision (philosophy, psychology). 
Integrative:®2 Synthesized from multiple perspectives into 
coherent integrated wholes (for example, philosophy, 
history, religion, etc.). 


Some disciplines fit into multiple realms, because there are different 
traditions within them. For example, psychology can be a way to make 
personal meaning, a way to make ethical meaning, or a rigorous empirical 
practice that makes meaning through expetiments and analysis. Even 
disciplines that seem to fit squarely within one realm can be crucially 


important to consider in other realms; for example, mathematicians 


81 In Realms of Meaning this was called “Synnoetic.” 
82 In Realms of Meaning this was called “Synoptic.” 
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would likely consider math as not only belonging to the symbolic realm, 
but also perhaps the integrative and even the aesthetic.” Rather than 
strict delineations, these categories provide a way of discussing high-level 
patterns in ways of producing knowledge. This abstract way of thinking 
about formulating questions or making decisions according to abstract 
values of a community is the exact kind of thing that computers cannot 


be trained to grasp. 


Problematic Knowledge 


The results of learning without meaning, or problematic knowledge, have 
been conceptualized in many different ways: fragile knowledge, rote or 
recipe knowledge, inert knowledge, and the prevalence of 
misconceptions. These types of problematic knowledge stem from 
slightly different ways of losing meaning in the learning process. 

Fragile knowledge, the most broad category, describes knowledge 
based on its behavior: it is not a good foundation, because when there is 
any stress added to it, it collapses and ceases to be useful. As the 
following figure shows, rote or recipe knowledge, is that which is based 
too heavily on doing, and not closely enough connected to conceptual 
understanding, such that students seem to be able to complete the 
activity that would require certain knowledge, but do not have a deep 
understanding that could result in high road transfer. Inert knowledge is 
the reverse; students seem to know some information when asked, but 


fail to apply it when necessary. 


Oo = : 
Q Zz Lack of 
~ Knowledge 
Low High 
KNOWING 


A deficiency of either knowing or doing results in problematic knowledge 
(recipe and inert, respectively). Source: CCR 


8 Lockhart, P. (2009). A Mathematician’s Lament: How School Cheats Us Out of Our Most Fascinating and Imaginative 
Art Form. Bellevue Literary Press. 
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If students simultaneously do not know about a concept abstractly 
and cannot execute a solution practically, this is simply a lack of 
knowledge, as shown in the bottom left corner of the figure. In order for 
students to adequately transfer knowledge from one situation to another, 
they need to both have a high understanding and a high ability to make 
use of their understanding. 

There is a multitude of popular pedagogical perspectives, all with the 
general underlying paradigm of constructivism,” and the general 
underlying goal of creating active-learning experiences for students. This 
balancing of top—down traditional learning (in which meaning is explicitly 
told to the students but may not effectively connect to student 
conceptions and experiences) with bottom-up progressive pedagogy (in 
which meaning is constructed by the students but may be limited in 
complexity) is an important part of the implementation of curriculum; if 
a particular curriculum focuses too much on the side of the top-down, 
the result will be inert knowledge, whereas if it focuses too much on the 
side of the bottom-up, the result will be recipe knowledge. Both of these 
types of knowledge do not transfer properly to new contexts for different 
reasons. The key here is to balance the need for both bottom-up 
approaches and top-down approaches, to create a learning experiences in 
which children create meaningful, useful, understandings. 

Misconceptions are simply understandings in which the meaning is 
not properly constructed. Often there ate counterintuitive 
understandings that a student must learn, but their original ways of 
making meaning of the world are more solidly established, and they 
inadvertently rely on them, instead of the more tenuously established 
complex ones they recently learned in school. Consider the following 
example that explores why students seem to drop robust conceptions of 


force mid-explanation. 


* The popular pedagogical paradigm created by Jean Piaget in which students actively construct 
understanding and cannot simply have it given to them. 
* These are often called “alternate conceptions” to preserve the idea that they still have meaning, they just do 


not match the standard. 
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... Thinking about the rising ball, students are likely to 
activate maintaining agency, the idea that effort must be 
continued in order to maintain the effect (“if you stop 
pushing, it’ll stop moving”). [The idea of] maintaining 
agency causes students to think that a continued upward 
“influence” must act on the ball to keep it moving upward. 
Asked about forces, students unconsciously map 
“influence” onto “force,” leading to an explanation that’s 
consistent with the motion requires force misconception. 
Thinking about the motionless peak of the trajectory, 
however, students’ intuitive sense of balancing turns on; an 
upward something seems to be balancing a downward 
something. Asked again about forces, students map that 
“something” onto force and say the forces balance.86 


Ultimately, how a student will interpret a given problem or situation 
will depend on how robustly they have constructed various 


understandings, and how the context triggers or fails to trigger these 
understandings to be used as tools for approaching the question. 


Optimization 


This helps explain the classic finding when it comes to forgetting 
knowledge: the Ebbinghaus forgetting curve. In 1880, Ebbinghaus 
published a paper (that has since been well replicated)” about the rate of 
forgetting, in which he showed that our memories had a steep decline 
and then levels off. 

The interesting thing to note here is the content that was being 
remembered. Since any idiosyncrasies in the matetial could throw off the 
results in unpredictable ways, these studies tested the retention of a string 
of nonsense syllables. If the goal was to design the materials so as to 
result in the most fragile possible knowledge, this is what would result, 
since the fact that they are nonsense allows no footholds for the brain to 
use, That makes this classic finding an interesting baseline measure for 


learning, but far from representative of learning. 


% diSessa, A. (1993) as cited in Hammer, D., Elby, A., Scherr, R. E., & Redish, E. F. (2004) in J. P. Mestre 
Œd.) Transfer of Learning from a Modern Multidisciplinary Perspective. LAP. 89-119, 
5 https://journals.plos.org/plosone/article?id=10.1 371/journal.pone.0120644 
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What would it look like to design material with the exact opposite 
goal: to provide as many footholds and routes of transfer as possible? 
There is a large opportunity for adaptive intelligent tutoring systems to 
help craft the ordering of material to optimize for transfer, both 


generally, and specific to each student. 


Core Concepts 


Any piece of information can serve as a tool for understanding any other 
piece of information. So how do we use the idea of knowledge being a 
tool as we design curriculum? The key lies in identifying and explicitly 
focusing on core concepts, the most powerful tools, both within each 


discipline and across disciplines. 


What is Most Important? 


The most complex aspect of curriculum design is to determine the most 
essential complex abstractions that should be taught to all students, 
within disciplines and across disciplines. 

There have been a number of efforts that have worked to highlight 
abstract concepts to help students build intuitions, make connections, 
and create more generalized understandings. Although they differ in 
framing and conceptualization, these frameworks are all pointing to the 
need for structuring curriculum to facilitate robust and flexible 


understandings via deliberate organization of concepts. 


Knowing and Doing 


As discussed in the previous section, transfer is really the process of 
knowing with an understanding that allows one to use or leverage 
previously learned information as a resource. It is thus no wonder that 
the distinction between knowing and doing has been of interest in 
cutticulum design. In the context of that dichotomy, knowing on its own 
is synonymous with having knowledge that one cannot necessarily 
transfer, or use in a new context, whereas doing involves necessarily 


mobilizing some knowledge to make something happen, even if that 
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knowledge is superficial and procedural rather than deeper and 
conceptual. 

It is not a coincidence, then, that various literacy” and fluency” efforts 
that extend these words beyond language have been increasing in 
popularity. After all, literacy is the ability to use your existing knowledge 
(of language, usually) to understand new information (by reading); in 
other words, to “know with” some set of information.” It thus conveys 
the idea of striving for expert amateurism and preparation for future 
learning.” Mastering the core concepts of a given discipline is the 
equivalent of being literate in that discipline, or “thinking like a 
[practitioner of that discipline].”””” Unfortunately, as discussed above, 
these concepts are so foundational to the thinking that they often go 
without saying, making them difficult to enumerate. 


Key Knowledge Frameworks 


The CCR framework is highly informed by those frameworks that came 
before it. Rather than reinventing the wheel, the goal was to create a 
framework that was as comprehensive as possible, while remaining as 
simple as possible. Many concept-based frameworks were reviewed, and 
the ones most key to the final synthesis can be found in the table that 
follows. 

What all of these frameworks have in common is a “less is more” 
approach to content, by placing concepts first. Instead of adding yet 
another element the curticulum must cover, it is consolidating content, 
structuring it more efficiently, leveraging transfer along the way.” As 
discussed, in order to facilitate transfer, a balance must be struck between 


eee 
88 For example, financial literacy, media literacy, scientific literacy, graphic literacy. 

® For example, math fluency. 

% To be fluent, a certain level of proficiency must be internalized to the degree of automaticity. 

*! Tt is even evident in our clichés, like when a scientist explaining something to a non-scientist who looks 
slightly annoyed and says “in English, please,” meaning that they are not fluent or literate, in this scientific 
topic. 

°? Wineburg, S., Martin, D., & Monte-Sano, C. (2014). Reading Like A Historian: Teaching Literacy in Middle 
and High School History Classrooms—Aligned with Common Core State Standards. Teachers College Press. 


3 An argument has been made that all reasoning/learning is at some level transfer (metaphor, analogy). 
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doing and knowing. The extreme form of doing results in recipe 
knowledge, in which one knows how to do something, but they ate not 
in fact leveraging deep understanding, rather they are simply 
remembering the superficial actions necessary to achieve their goal. 
Therefore, many of these frameworks explicitly try to pair knowing and 
doing in order to get to the heart of useful knowledge. Similarly, being 
able to think outside the scope of one discipline entails transfer to a new 
context, and is therefore also often noted in knowledge frameworks. The 
following table summarizes how the various knowledge frameworks 
relate to each other within the two dichotomies of knowing versus doing 
and disciplinary versus non-disciplinary.™ 

For example, a type of essential question in the disciplinary, big 
picture category is one that deals with what is foundational: these point 
to big ideas and frontiers in a discipline: “How many dimensions are 
there in space-time?” 

A type of essential question in the disciplinary, small picture category 
is one that deals with that is necessary for deep dives into content: “In 
what ways does light act wavelike?” 

A type of essential question in the non-disciplinary, big picture 
category is one that deals with what is timeless. These questions are 


interesting to debate and continue to change, such as, “What is justice?” 


4 We use the word “non-disciplinary” as opposed to interdisciplinary, cross-disciplinary, or transdisciplinary 
to avoid a discussion about exactly which kinds of connections outside the traditional confines of disciplines 


we mean. It is intended in the very general sense. 


Artificial Intelligence in Education 


g 
5 
Cd 
2 
a. 
T 
£ 
(2) 


Big Picture 


Small Picture 


Big Picture 


Big Ideas? 


Essential Questions? 


Theories? 


Strategies? 


Core Tasks? 
Processes” 


Essential Questions 


Principles? 


Subset Skills? 


Core Tasks" 
Practices? 
Transdisciplinary® 


Microconcepts® 
Generalizations” 
Macroconcepts? 
Concepts® 
Cross-Cutting Concepts 


Central ideas® 
Lines of Inquiry 


Representative Ideas® 
Threshold Concepts 
Essential Questions? 


Disciplinary Core Ideas’ 
Enduring Understandings? 


Disciplinary 
Disciplinary 


© Center for 


Curriculum Redesign 


g by Design (a), Concepts Based Education (b), international 


Baccalaureate (c), Next Generation Science Standards (d}, Realms of Meaning (e), Meyer and Land (f), and Rubicon (g). Source: CCR 
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Separating knowing and doing into a dichotomy may not actually be 
productive in the construction of a curriculum. After all, neglecting either 
one results in fragile knowledge. We therefore do not include this 
distinction in our framework, and simply focus on designing a process to 
determine which concepts and content to teach, and how to organize 
them. Truly learning a concept necessarily entails both the knowing and 


doing aspects. 


Concept Inventories as a Tool 


Attempts to start collecting, organizing, and assessing the deeper 
concepts for each discipline have also emerged. At the college level, there 
have been various efforts to create concept inventories for the sciences 
to assess how well students are learning the key concepts of a discipline. 
A concept inventory is created by collecting input from experts and 
educators of a given discipline, and constructing multiple-choice tests 
that diagnose students’ particular conceptual structures using distractor 
answets to identify common misconceptions. Look at the following 


example question” and its corresponding answer choices to get a sense. 


Imagine that you are an ADP molecule inside a 
bacterial cell. Which best describes how you would 
manage to “find” an ATP synthase so that you could 
become an ATP molecule? 


This question is designed to test whether students 
understand that diffusion is caused by random motion of 
molecules. 


a. I would follow the hydrogen ion flow. 


Students who choose this answer think that ADP 
somehow can identify where a hydrogen ion gradient is. 


b. The ATP synthase would grab me. 


Students who select this answer think that an ATP 
synthase senses the presence of ADP and actively grabs it. 


* Garfield, J., & Ooms, A. (2006). Assessment Resource Tools for Assessing Students’ Statistical Literacy, Reasoning, and 
Thinking, Proceedings of the National STEM Assessment Conference. 
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c. My electronegativity [is the main factor that] would 
attract me to the ATP synthase. 


Students who select this answer think that charges cause 
the ADP and ATP synthase to be attracted to each other. 


d. I would be actively pumped to the right area. 


Students who select this answer think that the ADP is 
somehow placed in the correct area so that it is close to 
the ATP synthase. 


e. Random movements would bring me to the ATP 
synthase. 


This is the correct answer. In other words, ADP finds 
ATP synthase by the random motion of ADP molecules. 

Through an intensive process of student interviews and iterative 
question development, these diagnostic tools are developed to be a 
diagnostic tool of student conceptions on a given topic. The first concept 
inventory was The Force Concept Inventory (FCI), created in 1992, By 
2008, twenty-three concept inventories had been developed in the 
sciences, and stirred discussion around the best teaching practices of 
higher education STEM subjects. 

We believe that this effort needs to be expanded in three ways to be 
effective. 


1. It should be used as a curriculum design tool, not just a 
diagnostic tool. 

2. It should include subjects outside of STEM. 

3. It should be adjusted (if necessary) for K-12 students. 


This expanded project would likely be most effective if conducted in a 
digital format, which would allow for continued growth, multiple 


contributors, interconnected organizational systems, etc.” 


°° Hestenes, D., Wells, M., & Swackhamer, G. (1992). “Force concept inventory.” The Physics Teacher 30: 141— 
158. 

*” One promising project is Ed’s Tools, which offers a computer program to help construct concept 
inventories for any subject. https://edstools.colorado.edu/ 
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Levels of Organization 


As concept-focused knowledge frameworks and concept inventories 
expand to become the basis for entire curricula, we must consider the 
hierarchy of concepts and how content are paired with concepts. Some 
of the frameworks that we drew from identified concepts that exist at 
different levels of content (for example, interdisciplinary, discipline 
specific, branch specific, or subject specific). We propose making this 
content/concept relationship explicit so students can most efficiently and 
effectively make meaning from the core concepts that exist actoss and 
within all K—12 disciplines. 


Structuring Content through Concepts 


As concept inventories expand to become the basis for entire curricula, 
they must be organized across different levels of content. If a piece of 
content is not connected to any concept in the curriculum, it will not be 
connected to a concept in the students’ minds, and will therefore not be a 
useful, transferable, piece of knowledge. Too often, courses are designed 
to hit the main content areas within a discipline to give students exposure 
to the range of subjects; however, organizing with this goal in mind will 
almost certainly lead to fragmented, disjointed content. 

But how can concepts be organized within disciplines and across 
them? The seminal Force Concept Inventory refers to a taxonomy that 
groups sets of items into categories, such as kinematics, impetus, and 
action/reaction pairs.” Others have referred to concept clusters,” use 


0 101 
ilea 


subtests,” mentioned macro and micro leve concepts, or explored 


interdisciplinary concepts such as maturity." Many frameworks also 


8 Hestenes, Wells, & Swackhamer, G., “Force concept inventory,” 141-158. 

» Steif, P. S. (2004). “An articulation of the concepts and skills which underlie engineering statics.” In Frontiers 
in Education, 2004. FIE 2004. 34th Annual (pp. F1F-5). IEEE. 

10 Evans, D. L., Gray, G. L., Krause, S., Martin, J., Midkiff, C., Notaros, B. M., & Streveler, R. (2003). 
“Progress on concept inventory assessment tools.” In Frontiers in Education, 2003, Vol. 1. IEEE. 

™ Kinchin, I. M. (2010). “Solving Cordelia’s dilemma: Threshold concepts within a punctuated model of 
learning.” Journal of Biological Education 44 (2): 53-57. 

102 Peter, M., Harlow, A., Scott, J. B., McKie, D., Johnson, E. M., Moffat, K., & McKim, A. M. (2014). 
“Threshold concepts: Impacts on teaching and learning at tertiary level.” Teaching > Learning Research Initiative. 
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mention the  inter-relationships among concepts. The exact 
conceptualization depends on the way each framework is organized, but 
there is general consensus that concepts are interrelated such that your 
understanding of one is affected by and affects your understanding of the 
others. 

We propose a structure that disentangles content and concepts, and 
identifies a handful of core concepts at each level of content organization 
(with many interdisciplinary connections). Topics are the lowest level of 
granularity and contain the content itself; they can be taught directly and 
should be used to exemplify, explore, and apply core concepts from 
across the higher levels. One of the advantages of this approach is that 
specific content (such as parts of a cell) can inherit high-level core 
concepts (such as scientific reasoning) and even concepts from other 
disciplines (such as division of labor). This organization helps curriculum 
designers keep track of the difference between learning objectives and 
the content used to teach them, facilitating a structure that revisits 
important concepts from different perspectives,” 


Content Structure 


Although there may be useful ways of organizing meaning in each 
discipline, it is ultimately not crucial to embark on the logistical challenge 
of reorganizing classes themselves. Rather, the keys to making meaning 
will be incorporated as core concepts at the various levels of 
organization. For many schools, a massive restructuring is not possible, 
and the CCR framework is designed to be as simple to implement as 
possible. í 

We therefore identify a set of names that correspond to groupings. 
Each one is made up of those below it (for example, disciplines are made 
up of branches). 


e  Disciplines—Mathematics, history, science, etc. 


° Branches—Algebra, Western civilization, biology, etc. 


16 This can be through a spiral curriculum, a strand curriculum, curriculum mapping, etc. 
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° Subjects—Game theory, the Russian revolution, ecology, etc. 
e Topics—Prisonet’s Dilemma, execution of the Romanovs, 


foraging, etc. 


It should be understood that no ontology and taxonomy can be 
perfectly coherent all the time and for all uses. For example, it may be 
useful to count both science and biology at a discipline level, therefore 
making two layers of the discipline type. At each level, the categories 
denote a particular grouping of content, and a particular set of defining 


core concepts. 
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Schematic of core concepts at each level of content organization (only 
topics can be taught directly, and thus must contain the core concepts of 
the higher level categories). Source: CCR 


Biology (a branch) as a grouping of content is the study of life and 
living things; its core concepts may be “Structure and function are 
interrelated,” “Natural phenomena tend to take the form of complex 
systems,” and so on. It is part of the scientific approach to knowledge, or 
science, (a discipline), which has its own core concepts, which may be 
something like “Scientific explanations, theories, and models are based 
on collective evidence and always contain a degree of uncertainty,” 
“Applications of science often have ethical, social, economic and political 
implications,” and “Science assumes that for every effect there is one ot 


> 


mote causes.” This focus on empirical and collective knowledge 
construction is distinct from, for example, the focus on beauty in the arts, 
or the focus on morals or narratives in the humanities. 

By assigning core concepts to each level in the taxonomy, a large and 


common mode of confusion is avoided. Curricula often choose just a 
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few of these levels and try to encompass all the relevant core concepts or 
big ideas within them, resulting in a redundancy, incompleteness, and a 
confusing mixture of levels. For example, an otherwise rigorous study of 
core concepts in physiology'* synthesized responses from faculty 
members to determine the core concepts of their field. Though the list 
was not initially structured based on how broad or narrow the core 
concept is, the following figure shows how we have organized these 
fifteen ideas into three content-based categories: the discipline, the 
branch, and the subject. 


1% Michael, J., & Mcfarland, J. (2011). “The core principles (‘big ideas’) of physiology: Results of faculty 
surveys.” Advanced Physiology Education 35: 336-341. 
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Science 
(Discipline) 


Causality: [Living 
organisms] are causal 
mechanisms (machines) 
whose functions are 
explainable by a 
description of the cause- 
and-effect relationships 
that are present. 


Scientific Reasoning: 
[Physiology is a science.] 
Our understanding of [the 
functions of the body] 
arises from the 
application of the 
scientific method; thus, 
our understanding is 
always tentative. 


Levels of Organization: 
Understanding 
[physiological functions] 
requires understanding 
the behavior at every level 
of organization from the 
molecular to the social. 


Mass Balance: The 
contents of any system or 
compartment in a system 
is determined by the 
inputs to and the outputs 
from that system or 
compartment. 


Categorizing physiology 
cote concepts into three 
layers of abstraction of 


content. Source. CCR 
adapted from Michael & 
Mcfatland 


Biology 
(Branch) 


Physics/Chemistry: The 
functions of living 
organisms are explainable 
by the application of the 
laws of physics and 
chemistry. 


Energy: The life of the 
organism requires the 
constant expenditure of 
energy. The acquisition, 
transformation, and 
transportation of energy 
is a crucial function of the 
body. 


Evolution: The 
mechanisms of evolution 
act at many levels of 
organization and result in 
adaptive changes that 
have produced the extant 
relationships between 
structure and function. 


Homeostasis: The 
internal environment of 
the organism is actively 
maintained constant by 
the function of cells, 
tissues, and organs 
organized in negative 
feedback systems. 


Interdependence: Cells, 
tissues, organs, and 
organ systems interact 
with one another (are 
dependent on the 
function of one another) 
to sustain life. 


Structure/Function: The 
function of a cell, tissue, 
or organ is determined by 
its form. Structure and 
function are intrinsically 
related. 
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Physiology 
(Subject) 


Cell-to-Cell 
Communication: The 
function of the organism 
requires that cells pass 
information to one 
another to coordinate 
their activities. These 
processes include 
endocrine and neural 
signaling. 

Cell Membrane: Plasma 
membranes are complex 
structures that determine 
what substances enter or 
leave the cell. They are 
essential for cell 
signaling, transport, and 
other processes. 


Cell Theory: All cells 
making up the organism 
have the same DNA. Cells 
have many common 
functions but also many 
specialized functions that 
are required by the 
organism. 


Genes to Proteins: The 
genes (DNA) of every 
organism code for the 
synthesis of proteins 
(including enzymes). The 
functions of every cell are 
determined by the genes 
that are expressed. 


Flow-Down Gradients: 


The transport of “stuff” 
(ions, molecules, blood, 
and air) is a central 
process at all levels of 
organization in the 
organism, and this 
transport can be 
described by a simple 
model. 


105 Flow-down gradients may be a lower level core concept than physiology. 
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By organizing core concepts according to levels, their number for 
each topic becomes manageable, as high-level concepts apply to all of 
their low-level elements. In this case, while the subject is physiology, 
there is no need to choose between scientific reasoning and flow-down 
gradients, because they are organized by scale. The other concept- 
focused frameworks only partially divide concepts based on the 
taxonomy of concepts. The following table shows a crosswalk that 
summarizes the way that CCR’s approach compares to similar concept- 


focused frameworks. 


54 


YOO :80uN0g 'saedwoo YOO Moy pue ‘js s}deou0e soR|d ey} BINoLUNO Jo yIEMSSOID 


{səaem pue spiel 


Branch Discipline 
TR ss 
Oo fo 8 
SNos 
oR & o 
SS Ü 
CES o] 
U5 
n v 
(do) 
D = O Q m 
‘ ® 2 UE 
G- gh Fa 2o93 
xe alk mM Q930Q > oO 
al O S O Ü DOr goad 
Bs QQQ5H 9 O° .x Oo F 
+5929 38 2 Sap Oo ie 
20002 9a 5 a 7 
SReOROR oe Q Q o 
eae ots | soe 8 5 
2959880 on a8ee 
eS SaB  VFRSSBo 
y > E e e 
Tao a@ 54 o< 3526 
=O ORD GS @25 
D DDS =i a a 
a a 
ot 
oe 
gear SoeF 
+ SOP F S52 OO 
98 2a o o2a G9 = 
p Fpa 2 AAL 
J. oQ 275390 
3282F 225390 
25 49S 323735388 
q0 2 Sno SË 
da fog BAG FR 
OLAS 6346» @ 
eee s 5 oaro d? 
2R a2 o 
ago 7 OSs 
Ces, 
am a35 47., 2E 
BESS 2S egars¢ 
ho. Se 2e% 98% 
iD ee 09 Zo Q x 
B2FQRB 2975 20 
EPSP RG o3 
E GamE nO o0 Uye 
G2Q8355 aS lse 
= ot 5 
sIr Sage 
R 03" zogggog 
5 09 5o g Qa a 
EA 
2 og 
aga 
oaao 5 
oga 
o RO 3G 
tr Qe Fy 
sag 
SSAA O 
2898 
a 83 40 
O O 
28808 
O Np 
RZ yw 
@ 
(youesg) (euljdiosiq) 


s}dəouo9 3109 


s}dəouop 2109 


peuyap Ajjeiouab ese s94 :suosəny JenUessy sos 


“Areuijdiosipsequl 4 


Artificial Intelligence in Education 


Interdisciplinary 


z 
w 
Q 
9 
Q 
Ö 
= 
6 
® 
QR 
(77) 
ioe) 
: : 
Q F 
X 9 a5 D 
£ a2a8 mM 
nS e R 
a c=[ 9 D 
9 oga 3 
es = 
ones Sees Q 
7) Oi ale 
O 3 o 
E Ta 
E RE TS 
eae O 
0O Qoy er 
o ogo 
~ 
2 3 
Q 
< S 
© = 5 
o 
seo ngo 
329 dQ 
O< 5 pao 
Q Q 
eSB 78e 
a es ae 5 20 
a Oo S ao 
eag Sod 
ea S85 
330g ZR 
Aa n D 
ae © 
Z 
O 
P| 
© 
” Q z 
fe) Q 
2803 28g 
fe) 
3 Q 0 9023 
On Si eae EOLO 
b03 300 
O 2 ogo A 
> OO AE 
0 FRO OG 
sor ER 
Cee E 
SLON OS 
2 5 Q gie 
ot fo) Q w 
cas Q. © 
ERE 952 
= 9 ays: 
"So org 
an . 
(ewey) 


sjdeou0d 3109 


eye ə1ƏUL 


Artificial Intelligence in Education 55 


Essential Content 


When asked what teachers hope their students take away from their class, 
attendees at CCR keynotes around the world as well as teachers in CCR 
seminars rarely mention specifics of content. In the real world, any 
particular piece of content is available for search instantaneously. This 
leads to the question: What content is essential to include in a 
curriculum? To answer this, we must first look at the purposes of 
content. 


If You Can Search Anything, Why Learn Anything? 


Learning happens in a context, and over time it is generalized and 
abstracted, but isolated information found online cannot fully capture the 
context in many cases. Daniel Willingham!” compares this to studying 
vocabulary words. Students are asked to use new words in sentences 
when they are learning them, in order to learn not just a definition, but 
how the word is used in context. When students simply look up 
synonyms online, they often end up using them incorrectly, such as 
saying “he meticulously balanced on the edge” (using the definition of 
“meticulous” to mean “careful”). The same reasoning, he argues, should 
be applied to all content learning. Just having the ability to look up a fact 
may not be enough to use and apply that fact properly. 


Avoiding the Dunning—Kruger Effect” 


One important use of knowledge is to guide us to what we don’t know, 
and should learn more about. As adults, there is a critical mass of 
knowledge that we use to create a rough map of our understanding and 
its gaps. Actor John Cleese humorously explains the Dunning—Kruger 
effect as “if you’re very, very stupid, how can you possibly realize that 
you're very, very stupid? You’d have to be relatively intelligent to realize 
how stupid you are.” Without a minimum understanding of a subject 


1% Willingham, D. (2017). “You still need your brain” Gray Matter. http://nytims/2rKoSPt 

107 Kruger, Justin; Dunning, David (1999). “Unskilled and unaware of it: How difficulties in recognizing one's 
own incompetence lead to inflated self-assessments.” Journal of Personality and Social Psychology. 77 (6): 1121- 
1134. 
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area, the trap one is likely to fall into is not just ignorance (which can be 


d 


cured with an internet search), but “meta-ignorance,” ignorance about 
one’s ignorance,’ which can be far more pernicious. 

For example, when reasoning about economic policies, citizens must 
implicitly estimate various economic realities in order to compare them 
to an ideal and consider possible changes. In a 2014 Gallup survey, 63% 
of Americans said they believed the crime rate was on the rise, despite 
the crime rate being at a 20-year low. Those who thought crime rates 
were rising were 8 percentage points less likely to support stricter gun 
control laws. 0”? 

If one doesn’t know that their estimates are not representative of 
reality, they may not think to look up the true numbers. In fact, in this 
study, participants who identified with both major US political parties 
misrepresented the inequality in the same way and agreed on the ideal 
distribution. Factual information, therefore, serves a crucial role in one’s 
ability to think critically and creatively. 

In fact, having the ability to look things up may exacerbate the effect. 
In one study,'"' participants who were allowed to use Google to answer 
trivia questions perceived themselves as smarter than did those who were 
not allowed to use Google (even when the percent of questions answered 


correctly was artificially equalized). 


Speed, Fluency, and Automaticity Used in Daily Life 


There is a basic level of each discipline that is necessary for day-to-day 
living. For example, there is a level of understanding that is necessary for 


basic math fluency with constructs such as weight, temperature, and 


108 Poundstone, W. (2016). Head in the Cloud: Why Knowing Things Still Matters When Facts are so Easy to Look Up. 
Little, Brown. 

10 Interestingly, 20 years prior, in 1994, people who thought crime rates were increasing were 9 percentage 
points more likely to support stricter gun laws, so there is some interaction with rhetoric. 

10 Kohut, A. (2015). “Despite lower crime rates, support for gun rights increases.” Pew Research Center. 
http://www.pewresearch.org/fact-tank/2015/04/17/despite-lower-crime-rates-support-for-gun-rights- 
increases / 

111 Wegner, D.M., & Ward, A,F. (2013). “The internet has become the external hard drive for our memories.” 
Scientific American 309 (6): 58-61. 


Artificial Intelligence in Education 57 


money. "? In neurotypical children this level is achieved without any 
concerted effort, but it is important to keep in mind which parts of the 


curriculum will be truly useful for all students’ lives. 


Part of a Shared Social Background 


Consider giving directions to a local compared to a tourist. When 
speaking to a tourist, we naturally understand that we cannot rely on any 
shared information or assumptions, and take much more time to explain 
things that we otherwise would take for granted." Similarly, news and 
media are not written in a way that explains every single idea; there is a 
collection of background information that is assumed and relied upon. 
E.D. Hirsch has worked to identify what content falls into this category 
for the U.S. (such as cholesterol, absolute zero) in his work on Cultural 
Literacy." Although this list would need to be adjusted for people from 
vatious sub-communities and other cultures around the world, it is an 


important exercise and useful starting point. 


Necessary for More Complex Concepts 


Every complex concept is made up of smaller pieces of information, 
which require automaticity to reach the more complex understandings. 
While anyone can look up anything at any time, having to look up 
everything would slow down future learning and problem solving. For 
example, although one could look up any unfamiliar words, this process 
is distracting from reading. Generally, the more vocabulary one knows, 
the greater their reading comprehension." This problem is exacerbated 


in settings where students have to process information in real-time, such 


112 Patton, J.R., Cronin, M.E., Bassett, D.S., & Koppel, A.E. (1997). “A life skills approach to mathematics 
instruction: Preparing students with learning disabilities for the real-life math demands of adulthood.” Journal 
of Learning Disabilities 30: 178-187. 

"3 Poundstone, W. Head in the Cloud. 

114 Hirsch Jr., E.D., Kett, J.F., and Trefil, J.S. (1988) Cultural Literacy: What Every American Needs to Know. 
Vintage. 

113 Schmitt, N, Xiangying J, and Grabe. W. (2011). “The percentage of words known in a text and reading 
comprehension.” The Modern Language Journal 95 (1): 26-43. 
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as lectures or group works, and do not have the option to look things up 
at their own pace when they need to. 

In such cases, a lack of fluency or automaticity (the combination of 
accuracy and speed) in low-level components can serve as a bottleneck to 
learning high-level concepts.''® More broadly, research has shown that 
fluency “increases retention and maintenance of skills and knowledge, 
endurance or resistance to distraction and application or transfer of 
training,” 

But learning a concept is usually not simply the process of amassing 
the smaller pieces of information that comprise it. This is the thinking 
behind the research exploring learning progressions: Moshet’s A 
Hitchhiker's Guide to Thinking about Literacy, Learning Progressions, and 
Instruction notes, “curricula should be designed to provide students with a 
systematic exposure to increasingly complex meanings ... and grounding 
them in experiences with particular content and topics”.'’* It may be that 
learning topics in a certain order or through a certain pathway will lead to 
the knowledge being represented and stored differently, and to serve as 
pteparation for different types of future learning. Therefore, another 
reason some knowledge may be included in a curriculum is that it is part 


of a particularly effective learning progression. 


Content as Substrate for Core Concepts 


If the majority of what teachers want students to take away takes the 
form of concepts that can be applied to new situations, there is still a 
decision to be made about what content best illustrates the concepts, 
such that students learn them and are able to transfer that understanding. 
The context in which a particular concept is learned deeply affects the 


eye . : 119 
way it is structured in the learnet’s mind. 


116 Binder, C. (1993). “Behavioural fluency: A new paradigm.” Educational Technology. 

7 Thid. 

118 Mosher, F. (2017). A Hitchhiker’s Guide to Thinking about Literacy, Learning Progressions, and Instruction. 
Consortium for Policy Research in Education. http://www.cpre.org/hitchhikers-guide-thinking-about- 
literacy-learning-progressions-and-instruction 


49 https://en.wikipedia.org/wiki/Situated_cognition 
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When content is organized in a curriculum, it is constructed as a 
progression with a scope and sequence. Based on this sequence, later 
elements can leverage understanding built earlier, and the curriculum can 
build to cover greater complexity over time efficiently. It is within the 
context of a curriculum that content elements take on a relative scope, in 
terms of what elements they tie together. According to Moshet’s 
Hitchhiker's Guide, “curricula should be designed to provide students with 
a systematic exposure to increasingly complex meanings ... and 
grounding them in experiences with particular content and topics.” The 
same paper says, “no particular order of instruction is necessary, but we 
argue that picking a reasonable order across a school or school system 
(and even more widely if possible) is wise, and likely to be more effective 
than leaving the choice solely to individual teachers.” 

Therefore, when a concept is first introduced, it should be through an 
exemplar” —content that naturally demonstrates the concept, such that 
the student can internalize a set of intuitions. This means it is not merely 


' it makes salient the features that are 


an example, but it exemplifies:'” 
relevant to the concept. The exemplar content should be stable, 
accessible, and have minimal distracting ptoperties.'” Since each topic is 
full of information and what is signal for one concept is noise for 
another, this step often involves some amount of setting the stage and 
orienting students to the relevant features. By seeing the same concept 
through different content, the elements that are part of the deeper 
structure and the elements that are incidental to the context become 
apparent.” 

After that, however, the concept should be explored in less 
straightforward contexts, with less scaffolding from the teacher. At this 


12 This aligns with the use of the term in psychology: https://en.wikipedia.org/wiki/Exemplat_theory 


121 Elgin, C. Z. (2017). True Enough. MIT Press. 

122 Thid. 

13 For example, Quilici, J.L., & Mayer, R.E. (1996). “Role of examples in how students learn to categorize 
statistics word problems.” Journal of Educational Psychology 88 (1): 144-161. http://doi.org/10.1037//0022- 
0663.88.1.144 and Tenenbaum, J.B., Kemp, C., Griffiths, T. L., & Goodman, N.D. (2011). “How to grow a 
mind: statistics, structure, and abstraction.” Science 331 (6022): 1279-85. 
http://doi.org/10.1126/science.1192788 
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stage, the particular topic might be an exemplar of a different set of 
concepts, but still be an instantiation of a concept that has already been 
introduced. As the word suggests, the topic is an instance of the concept, 
but being just one instance of many, it also has many other features. In 
other words, the topic is an example of the concept, but not an exemplar. 

If the color red has been covered in the context of the color wheel as 
an exemplar, it may be useful to point out all of the things on the street 
that are red, (even though this topic—the reason you are on the street— 
may be the exemplar for learning cardinal directions). That the concept 
may not be salient (camouflaged, occluded, overshadowed, or a 


borderline case’) 


allows students to practice generalizing the concept 
and fine-tune their understanding. 

This structure naturally makes use of one of the most robust findings 
of the learning sciences: interleaving information that should ultimately 
be integrated leads to more long-term learning than studying the 
information sorted into its natural categories." Normally, this is applied 
to content, but in this case the explicit untangling of content and 
concepts leads to the inevitable grouping by content and thus interleaving 
of concepts.’ 

Finally, the concept can be an application to a conceptually distant 
topic. The challenge may be that the teacher does not explicitly prompt 
the application of the concept, more closely simulating real life scenarios, 
and/or that the concept is not salient in the context. In the example of 
teaching the concept of the color red, the application topic may not 


12 “Camouflaged: a tiger who blends into the surrounding jungle instantiates being striped but would in that 
circumstance be unlikely to exemplify stripedness. (It might, however, exemplify how stripes 


even orange 
stripes—can camouflage in a jungle of green.)” “Occluded: a bald man wearing a hat is ill positioned to 
exemplify his baldness.” “Overshadowed: the fearsome timbre of a lion’s roat may block its effectively 
exemplifying its pitch.” “Borderline case: even though a chartreuse fire hydrant is a vivid instance of 
greenness, its color is too close to yellow to be a good exemplar.” In Elgin, C. Z. (2017). True Enough. MIT 
Press, 1—91. 

13 Rohrer, D. (2012). Interleaving helps students distinguish among similar concepts. Educational Psychology 
Review, 24(3): 355-367. 

1% Those who have had experience working with Spiral Curricula may recognize that this framework would fit 
well with the notion of revisiting concepts, but that is just one way of conceptualizing this underlying 


structure. 
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include anything that is red, but the student can learn to see that red is 
part of what makes orange and purple, which are in the application topic. 
By interweaving contexts that make salient different concepts, a scaffold 
for conceptual understanding can be constructed that enables complex 
concepts to be built up simultaneously and systematically, in a way that is 
likely to transfer to real life contexts. While this is the same process by 
which experts eventually gain this conceptual understanding, there is no 
reason that it can’t also be explicitly and systematically used as a 
curriculum-design guideline for students. 

This idea is discussed in The Inquiry Project in the context of always 
teaching more than one concept at a time: “One always is considering 
portions of several concepts (foregrounding some, backgrounding 
others), working on successive subconcepts, such as scale weight, each of 
which involves relations among parts of concepts, revisiting concepts and 
amplifying the subconcepts and contexts considered.” 


Knowledge as Substrate for Competencies 


Although we have not discussed competencies in detail in this book 
(please see Four-Dimensional Education: The Competencies Learners Need To 
Succeed” for further details), knowledge cannot be discussed completely 
in a vacuum. The perceived importance of teaching students how to 
think, how to learn, and how to apply socio-emotional skills, often 
overshadows discussions about which content should be taught in the 
first place. However, it is important to note that learning is highly 
situated and context-dependent, and competencies are thus best taught 
through some fitting content substrate, or medium.'” And there is reason 


127 Wiser, M., Smith, C.L., Doubler, S., & Asbell-Clarke, J. (2009). “Learning progressions as a tool for 
curriculum development: Lessons from the Inquiry Project.” Paper presented at the Learning Progressions in 
Science (LeaPS) Conference, June 2009, Iowa City, IA 

128 Fadel, C., Bialik, M., and Trilling, B. (2015). Four-Dimensional Education: The Competencies Learners Need to 
Succeed. Center for Curriculum Redesign. 

12 Garner, R. (1990). “When children and adults do not use learning strategies: Toward a theory of settings.” 
Review of Educational Research 60 (4): 517-529. 
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to believe that some content may be better aligned for teaching some 
competencies than others. 

For example, it is unclear whether mathematics is a good vehicle for 
teaching critical thinking (as is often supposed) since the same features 
that make it very rigorous when mastered, make it difficult to grasp and 
thus practice for novices." It may be that a more concrete domain, such 
as social studies, and a more accessible learning mechanism, such as 
debate, would be more effective. This points to the need for further 
research into proving/disproving the explicit and implicit claims of 
vatious disciplines regarding their role in the development of cognitive 


competencies. 


Modernized Knowledge 


When choosing content to include in a curriculum, it is important to 
make sure that it is not outdated. This is challenging for a few reasons. 

First, consensus on facts changes as more information is gathered, 
and observing the aggregate scale, there can be said to be a predictable 
rate of decay of facts within a given discipline.” Therefore, even if one 
could learn all of the most relevant facts, as their life continues, a growing 
fraction of that knowledge will become incorrect. Learning is never 
finished; it is always setting up a foundation for future learning. 

Second, the types of jobs available are changing. Forecasting new 
professions has always been a difficult endeavor since the tendency to 
think linearly prevails, and discontinuities in progress, and thus needs, 
cannot be forecast. Some accounts conclude that automation is 
fundamentally changing the workforce by eliminating jobs, while others 


conclude that automation creates almost as many jobs as it replaces. 


130 Lehman, D.R., Lempert, R.O., & Nisbett, R.E. (1988). “The effects of graduate training on reasoning: 
Formal discipline and thinking about everyday-life events.” American Psychologist 43 (6): 431-442. 

131 Bialik, M., & Kabbach, A. (2014). “Mathematics for the 21st Century: What should students learn? Paper 4: 
Does mathematics education enhance higher-order thinking skills?” Center for Curriculum Redesign. 

132 https://en.wikipedia.org/wiki/Half-life_of_knowledge and, in particular, Thierry Poynard’s article in 
Annals of Internal Medicine 136: 888. 
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Regardless, the types of jobs available to workers worldwide look 
different today than they did 20 years ago.'” 

Looking at the change in the types of jobs held by workers from 
1995-2015 in the figure below, the polarization of jobs is apparent. The 
number of high-skill jobs (i.e. managerial and professional) and to a lesser 
extent low-skill jobs (service and retail) is increasing while middle-skill 
jobs (trade work, machine operation, and assembly) are disappearing. 
Given these trends, there is ever-greater urgency to help students to build 
a strong foundation for complex content areas, and to prepare them for 


the situation that they will need up-skilling during their lifetimes. 
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Job polarization by country: Percentage point change in share of total 
employment, 1995 to 2015. Across all countries, the number of middle- 
skill jobs (gray) decreased by an average of about 10%. High-skill jobs 
(dark gray) increased by about 8% and low skill jobs (light gray) 
increased by about 2% on average. Source: OECD 


Specifically, tasks that involve predictable physical work, data 
processing, and data collection are the most susceptible to being 
automated, while also accounting for about 50% of all work activity in 
the US."* The jobs least likely to be automated are those that require 


expertise, interacting with other people, and in particular, management 


1833 OECD. (2017). OECD Digital Economy Outlook 2017, OECD Publishing. 
http://dx.doi.org/10.1787/9789264276284-en 

134 Chui, M., Manyika, J. and Miremadi, M. (2016). “Where machines could replace humans—and where they 
can’t (yet).” McKinsey Quarterly. https://www.mckinsey.com/business-functions /digital-mckinsey/out- 


insights / where-machines-could-replace-humans-and-where-they-cant-yet 
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skills. This makes intuitive sense, since algorithms will take over the most 
rote tasks first, people will need to manage these computers just as they 
managed those tasks when humans were performing them. 
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Top chart: Percent of time spent on activities that can be automated by 
adapting currently demonstrated technology. Bottom chart: Percent of time 
spent in all US occupations. Source: McKinsey 


Third, it is important to consider the larger trends taking place. The 
world is changing at an increasing rate along multiple fronts: technology, 
environment, globalization, and advances in our understanding, which 

| ought to change the content that should be included in the curriculum. 
| Ideally, education reform would minimize the lag between human 
progress and preparation for people to enter the changing landscape of 
the workforce, society, and life. Each of these will require different 


content areas to be added to the now outdated curriculum, and various 


updates to be made to the focus of what students ought to take away 


from those content areas. 


These trends will affect what is important in the entire range of ways 
of making meaning and can be summarized into the following major 
changes: 
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1. Technology: Trends that are forecast to be particularly 
transformative are: '*° 
a. The rise of smart machines and systems (robotics ete.) 
b. Massive data and new media (big data, social 
networks/media, etc.). 
c. Amplified humans (artificial intelligence, robotics, gene 
editing, etc.). 

2. Environment: Climate change is a new and large-scale challenge, 
whose stresses and demands must be addtessed.'° 

3. Globalization: People and organizations are increasingly 

interconnected, and creating previously unforeseen emergent 
patterns.” 

4. Social Unrest: With greater inequality, polarization, distrust and 
large-scale changes, there is global social unrest that must be 
addressed. 

5. Advances in the disciplines: Curriculum often fails to include new 
findings due to already being full; it should be updated to include 
the important recent discoveties. 

One way to capture these changes is through the six realms of 
meaning mentioned earlier. Each of the realms roughly encapsulates a set 
of disciplines, making it a simple way to make sure that the full range of 
content is considered. Cross-referencing these two sets of categories 
produces a table that illustrates the types of topics or disciplines that 
need to be emphasized or added wholesale. The following table illustrates 


some examples at their appropriate intersections. 


135 Davies, A., Fidler, D., Gorbis, M. (2011). “Future work skills 2020.” Institute for the Future. 
136 Tbid. 
137 Ibid. 
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138 Botsman, R. (2017). Who Can You Trust: How Technology Brought Us Together and Why It Might Drive Us Apart. 
PublicA ffairs. 
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What Content Should be Added 


In order to reflect the changes that are occurring in society, it is 
important to add modern disciplines, branches, subjects, and topics 
where appropriate, and not be constrained by inertia. Currently, these 
disciplines have been crowded out by the traditional disciplines, and yet 
they are becoming increasingly useful and deserving of space in the 
curriculum. Below is a (non-exhaustive) list of modern content areas that 
should be integrated into curriculum, as they are now widespread in use 


and in importance. 


Technology and Engineering 


This includes computer science, in particular: coding, robotics and 
attificial intelligence; bioengineering, in particular, genome editing and 
synthetic biology; as well as advanced manufacturing including CAD, 3- 
D printing. 


Media 


The changes that the internet has made to society have just begun, and 
already they have been transformative. Everyone constantly consumes 
and creates media, and this has become inextricable from the social 
world. And yet, no one is taught how to properly and healthily use this 
media. Curricula must change to keep up. This includes journalism, 
including the different forms it is now beginning to take, and 
audio/video. If everyone can make a video about anything, what does 
this mean to interact with this world in a productive way? How can one 


discover their voice? 


Entrepreneurship and Business 


As jobs continue to become more polarized by the skill levels they 
require, as the landscape of what is needed in the workforce continues to 
change, and as the economy nevertheless continues to grow, students will 
need to be prepared to take advantage of opportunities that present 
themselves throughout their lives. It is often no longer enough to find a 
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job out of college and work there until retirement. Students must be 


prepared to approach their careers from a business perspective. 


Personal Finance 


As jobs become less stable and opportunities become more varied, while 
the laws are becoming more complex, and personal debt continues to 
grow, those entering the workforce need to have some preparation on 
how to organize their financial lives. This is perhaps the most applicable 
knowledge students will learn, and is relevant to all students. 


Wellness 


Rather than only treating problems when they become a serious issue, 
students should be taught how to proactively take care of themselves 
mentally, emotionally, and physically. This has come to be known as 
wellness and is a very broad category, and often spanning some sort of 
space from physical education to health education. In a society with 
problems like depression, anxiety, obesity, and chronic back pain, 
students should be informed and empowered to keep themselves well. 
They should also be taught about interpersonal wellness—what 
constitutes emotional or psychological abuse, how to navigate 
relationships, and be given practice developing their own foundation of 


emotional awareness. 


Social Sciences 


The social sciences study humans as their topic, but use systematic 
methods to do so. This may include sociology, anthropology, psychology, 
political science, future studies, civics, etc. This area of research has not 
only made great advances and become very important in many 
professions that deal with humans, but it also entails an important and 
unique way of making meaning of the social world. The world students 
need to be prepared for is increasingly interpersonal, and the social 
sciences address these topics. 

Naming the disciplines, branches, subjects, and topics that may 


become more relevant due to the trends in progress is only the first step. 
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As this material gets integrated it will be necessary to identify the core 
concepts and an effective scope and sequence through them. It is also 
important to note that while modern disciplines need to be added, this 
does not mean that traditional disciplines must be abandoned. Adding 
modern disciplines can be an act of consolidation, as the important 
concepts from previously separate subjects may unite new and old 
material, while strengthening and deepening the potential for 


understanding, 


What Content Should be Removed 


The method of consolidating does, however, also mean that some 
content must be removed. The redesigned curriculum should be more 
efficient, doing more with less, as discussed above. In many ways, the 
curriculum has been extremely stagnant, even since ancient times (See 
appendix 1). It is an important part of the redesign process to comb 
through the content of the curriculum and remove anything that is 
obsolete, or redundant and not an efficient use of time. This is often the 
hardest part, since it may feel risky to remove something that has been 
present for so long and is present in so many other curricula (and when 
done well, may be useful as an example of a concept). According to 
Perkins, “Curriculum suffers from something of a crowded garage effect; 
it generally seems safer and easier to keep the old bicycle around than to 
throw it out.”’*” But the point of the curriculum, as discussed, is not to 
covet as much content as possible. Without this process of removal, 


change cannot happen. 


Non-Disciplinary Structures 


So far the discussion of curriculum structure has been under the 
assumption that disciplines are relatively distinct, as is their teaching. 
However, this does not have to be the case. 

Disciplines have been a convenient way of organizing knowledge, but 


if change is to happen and deep, relevant, transferrable concepts are to be 


139 Perkins, Future Wise. 
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learned, it is likely that a less discipline-focused approach will be needed. 
After all, if the most important things about knowledge are its relational 
qualities—mapping ideas and activating resources—more relationships 
across mote disciplines will be a powerful way to bolster mental models 
of concepts (as well as consolidate content). For instance, exponentials 
(from mathematics) can be taught alongside compound interest (from 
finance) and financial bubbles (history, sociology), as well as bacterial 
growth (biology) and resource exhaustion (environmental literacy). 

Additionally, interdisciplinary approaches can be a powerful 
motivational tool, as students may explore different contexts, meanings, 
and applications of concepts, and find angles they are personally curious 
about. This allows students to be guided by intrinsic motivation, or 
interest, rather than extrinsic motivation like grades or performance. 

A report issued by the National Academy of Sciences” in the United 
States identifies four primary drivers of interdisciplinarity today: 


1. The inherent complexity of nature and society. 

2. The desire to explore problems and questions that are not 
confined to a single discipline. 

3. The need to solve societal problems. 


4. ‘The power of new technologies. 


Accordingly, the interest in interdisciplinarity and its analogues has 


rapidly emerged since the 1960s. 


Trends in articles with the term “interdisciplinary” in the title 1990-2007. 
Source: Jacobs, Jerry, and Frickel 


40 National Academy of Sciences (2004). Facilitating Interdisciplinary Research. Washington, DC: National 
Academies Press, 2, 40. 
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Interdisciplinary studies is now a major area of study, and interest in it 
continues to grow as the preceding charts show. In order to get a more 
clear sense of the entire landscape of knowledge and how disciplines 
relate to one another fluidly, it may be possible to make use of new 
analysis and imaging tools, such as what is discussed in the first half of 
this book. 


Embedding Interdisciplinary Themes in Curriculum Design 


Running through disciplines are crosscutting themes, which can be drawn 
from as a source of examples for any disciplinary content. These can be 
applied to any other body of knowledge, as a way of seeing it in a 
particular perspective with its particular focal points. 

As discussed earlier, these are often thought of as different types of 
literacies, which naturally convey the ideas of “knowing with” and 
preparing for future learning. In this sense, they ate at the same scale as 
disciplines, but inherently trans-disciplinary. Like disciplines, they each 
have their own set of core concepts, which encapsulate the habits of 
mind that each theme comprises. CCR has identified the following as 
fruitful themes (See appendix 1 for a deeper discussion of each): 


e Environmental Literacy e Digital Literacy 

e Global Literacy e Systems Thinking 

e Civic Literacy e° Design Thinking 

° Information Literacy e Computational Thinking 


These are simply important categories to consider when designing 
curticulum to develop knowledge that will be useful across disciplines. 
Like disciplines, each lens would have its own set of core concepts that 
students would be expected to internalize, through a range of different 


soutces of content. 


Changes in Disciplines 


The exact connections and lines between disciplines are always shifting, if 


slowly. Over time, disciplines branch to create subfields and merge to 
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create interdisciplinary fields. For example, in the following image,'*' the 
field of neuroscience emerges as a synthesis of molecular & cellular 
biology, psychology, and neurology, whereas urology emerges as a 
subfield that splinters off of medicine. 


2005 = 


The emergence of neuroscience as a discipline. 
Source: Roswall, Martin, and Bergstrom 


By disregarding the directionality of time, all disciplines can be 
conceived of as interdisciplinary, either because they are at the root and 
thus contribute to other disciplines,'” or because they are composites of 
other disciplines. Newer disciplines tend to be in some ways composite 
of older disciplines (for example, optogenetics comprises optics, 
neuroscience, and genetics) or they may be sub-disciplines, splintering off 
from existing ones (such as civil engineering). 


1 Rosvall, M., and Bergstrom, C.T. (2010). “Mapping change in large networks.” PS ONE 5.7. 
https://journals.plos.org/plosone/article?id=10.1371 /journal.pone.0008694 
142 Crease, R.P. (2010). “Physical Sciences.” In The Oxford Handbook of Interdisciplinarity. 
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Interdisciplinarity in all disciplines, root and composite. 
Source: CCR 


Curricula tend to focus on root disciplines (especially under pressure, 
such as assessments that test root disciplines in isolation). The goal then 
becomes to teach more of the newer interdisciplinary subjects, and to 
highlight the interdisciplinarity in the root disciplines. 

Interdisciplinarity is likely to become more viable as cutticula place 
concepts at the center, and content is designed to serve those mote 
abstract elements, to achieve transfer. In that case, examples can be 
chosen deliberately, preparing students for learning across disciplinary 


lines. 
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Practical Considerations 


There are three options for how to redesign the curriculum’s treatment 


of disciplines: 


1. Traditional: Disciplines are organized into departments 
directly, and modern disciplines are simply added either as 
units to their related discipline, or as an elective. 

2. Hybrid: Disciplines are intact, but the departments are novel 
categorizations. These can be Themes or Realms of Meaning 
as discussed earlier in the document. 

3. Redesigned: Disciplines are no longer the organizational 
backbone of the curriculum and classes are mostly on hybrid 
topics, such as systems thinking. This makes adding new 


disciplines natural. 


The route each school chooses to go will be a result of their goals, 
capabilities, buy-in, pressures, and so on. Even a traditional 
organizational structure can go quite a long way toward teaching 
interdisciplinarity through projects and the interconnectedness of core 
concepts. However, the exact organizational structure is not as important 
as aiming for a robust and flexible understanding of the fundamentals, by 
identifying the essential content and core concepts and structuring 


learning experiences around them. 


Decision Making 


One of the main tensions with organizing curriculum is the level at which 
the various decisions are made. At the highest level are disciplinary 
experts and policymakers, followed by curriculum designers, then 
teachers, and finally students. It is possible to completely prescribe all 
elements, or to leave all elements completely up to the students, and of 
course, most organizations fall somewhere in between. The particular 


configuration of responsibility depends on the constraints and 
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preferences of a given jurisdiction, but they should be made explicit'® 
and used as a guideline for appropriate division of labor and 


corresponding expectations and communication channels. 


Conclusion 


Once the levels of core concepts have been established, the particular 
pieces of content must be considered for inclusion or exclusion, one by 
one." There should be no content that does not serve concepts. Most 
content will exemplify a specific concept that has a high potential for 
being leveraged to understand many aspects of the world. It should also 
setve as a good instantiation or application of concepts that are 
exemplified elsewhere. 

In addition to serving concepts, content across all levels of primary 
and secondary education should be: 


1 Modernized with the addition of new disciplines and new 
approaches to old disciplines, including interdisciplinarity. 

2: Systematically increasing in complexity so that as students learn 
more they ate asked to make more and broader.connections to 
increasingly abstract (but relevant) concepts. 

3: A substrate for teaching competencies which include skills, 


character, and meta learning. 


The goal is to rid the curriculum of obsolete, irrelevant information, 
while simultaneously modernizing, systematically sequencing, and 
infusing the content with competencies. Students should learn the useful 
ways of seeing the world developed by different disciplines, as well as 
particularly important topics and discoveries within and across the 
disciplines. In other words, the goal is to maximize the potential for 
making meaning in the curriculum. This is not just so that students find 


school more enjoyable, but also to make their learning more useful to 


18 Bialik, M. & Fadel, C. (2017). “Overcoming system inertia in education reform.” Center for Curriculum 
Redesign. https://curriculumredesign.org/wp-content/uploads/Inertia-in-Education-CCR-Final.pdf 


14 In practice, this process will likely be more iterative than we suggest here. 
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them later in life. Without meaning, a structure of understanding is not 
built, and learning does not get internalized in a way that applicable 
outside of its original context. 

This idea can be exemplified through an analogy to learning to play an 
instrument. There is nothing wrong with beginning with drilling scales, as 
long as more meaningful learning takes place not long afterwards. It is 
possible to get sucked into the minutiae of music and never get to the 
meaning of it, just as it is possible to get sucked into any other minutiae. 
In this excerpt from A Mathematician’s Lament,” a dystopian world is 


described where music is taught the way math is taught now: 


Since musicians are known to set down their ideas in the 
form of sheet music, these curious black dots and lines 
must constitute the “language of music.” It is imperative 
that students become fluent in this language if they are to 
attain any degree of musical competence; indeed, it would 
be ludicrous to expect a child to sing a song or play an 
instrument without having a thorough grounding in music 
notation and theory. Playing and listening to music, let 
alone composing an original piece, are considered very 
advanced topics and are generally put off until college, and 
more often graduate school. As for the primary and 
secondary schools, their mission is to train students to use 
this language—to jiggle symbols around according to a 
fixed set of rules: “Music class is where we take out our 
staff paper, our teacher puts some notes on the board, and 
we copy them or transpose them into a different key. We 
have to make sure to get the clefs and key signatures right, 
and our teacher is very picky about making sure we fill in 
our quarter-notes completely ...” 


To avoid mistreating other subjects like this, it is important that each 
piece of curriculum is there for a reason, and that meaning is never too 
far away. Students can continue to practice scales and notation, even as 
they begin to play solo pieces, play pieces as part of an orchestra, 
compose their own pieces, and even learn to improvise. Ultimately, they 
are nevet going to need to play a perfect scale (or the equivalent in 


145 Lockhart, P., A Mathematician’s Lament. 
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another subject) but they are going to rely on the automaticity of their 
scales in order to do higher level processes such as composition and 
improvisation. It is therefore crucial that meaning is the primary concern, 
and all else is in service of that, as that is what will effectively get 


internalized and prepare students for the future, whatever it may hold. 
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Part Two 
The How: Promises and Implications of AI 
for Teaching and Learning 


Seldom a day goes by without at least a mention in the news or 
entertainment media of AI. Perhaps an AI program has just beaten the 
world’s leading player in a complex strategy game; perhaps a new 
Hollywood feature film depicts a dystopian future in which robots have 
overtaken the world; or perhaps a pair of leading tech entrepreneurs have 


a public disagreement.'*° 


I have exposure to the very cutting-edge AI, and I think 
people should be really concerned about it. ... AI is a rare 
case where we need to be proactive about regulation 
instead of reactive. Because I think by the time we are 
reactive in AI regulation, it’s too late. 

—Elon Musk 


I think people who are naysayers and try to drum up these 

doomsday scenarios ... I just, I don't understand it. It's 

really negative and, in some ways, I actually think it is 

pretty irresponsible. 

—Mark Zuckerberg 
In fact, as the exchange between Mark Zuckerberg (Facebook) and 
Elon Musk (SpaceX, Tesla) suggests, the future impact of AI remains 
very unclear (indeed, is artificial intelligence little more than the latest 
technical hype?).'*’ Nevertheless, investments and developments 
continue to grow exponentially, such that AI has become an integral, 


pervasive and inescapable, although often hidden, part of our daily lives: 


16 E.g., https://www.nytimes.com/201 8/06/09/technology/elon-musk-mark-zuckerberg-artificial- 
intelligence.html 

197 “Highly-publicized projects like Sophia [http://www.hansonrobotics.com/robot/sophia] [try to] convince 
us that true AI—human-like and perhaps even conscious— is right around the corner. But in reality, were 
not even close. The true state of AI research has fallen far behind the technological fairy tales we’ve been led 
to believe. And if we don’t treat AI with a healthier dose of realism and skepticism, the field may be stuck in 
this rut forever.” Dan Robitzski quote (2018) at https://futurism.com /artificial-intelligence-hype 
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from Siti'** to auto-journalism,'” from forecasting stock movements'” to 
predicting crime,'*' from facial recognition” to medical diagnoses” and 
beyond. 

But of particular interest here, artificial intelligence has also quietly 
entered the classroom.’* Whether students, teachers, parents and policy 
makers welcome it or not, so-called intelligent, adaptive, or personalized 
learning systems ate increasingly being deployed in schools and 
universities” around the world, gathering and analyzing huge amounts of 
student big data, and significantly impacting the lives of students and 
educators.'°° However, while many assume that artificial intelligence in 
education (AIED) means students being taught by robot teachers, the 
reality is more prosaic yet still has the potential to be transformative. 
Nonetheless, the application of AI to education raises far-reaching 
questions. 


We should ask what happens when we remove care from 
education... What happens to thinking and writing when... 
the whole educational process is offloaded to the 
machines—to “intelligent tutoring systems,” “adaptive 
learning systems,” or whatever the latest description may 
be? What sorts of signals are we sending students? 

i —Audrey Watters!5’ 


148 https://www.apple.com/uk/ios/siti 

49 E.g., https://www.washingtonpost.com/pr/wp/2018/06/12/the-washington-post-plans-extensive- 
coverage-of-2018-midterm-elections/?utm_term=.e66d88e4a716 

150 E.g., https://equbot.com 

151 E.g., http://www.predpol.com 

wEB, https://www.cbp.gov/newsroom/national-media-release/cbp-deploys-facial-recognition-biometric- 
technology-1-tsa-checkpoint 

153 Eg., https://www.babylonhealth.com 

154 Luckin, R., et al. (2016). Intelligence Unleashed. An Argument for Al in Education. Pearson. 
https://www.pearson.com/content/dam/one-dot-com/one-dot-com/global/Files/about- 
pearson/innovation/Intelligence-Unleashed-Publication.pdf 

155 “The time-to-adoption for adaptive learning technologies and artificial intelligence is estimated within two 
to three years, acknowledging the advances in these technologies and their promise to positively impact 
teaching and learning.” Becker, S.A., et al. (2018). “Horizon Report: 2018.” Higher Education Edition 2. 

156 Holmes, W., et al. (2018). Technology-Enhanced Personalised Learning. Untangling the Evidence. Robert Bosch 
Stiftung. 

157 http://hackeducation.com/2015/08/10/digpedlab 
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In fact, AI technologies have been researched in educational contexts 


for around fifty years. More recently, companies as influential as 


Amazon, Google and Facebook have invested millions of dollars” 
developing AIED products, joining well-established multimillion dollar- 
funded AIED companies such as Knewton'® and Carnegie Learning,'*! 
while the $15 million Global Learning XPrize'® called for software that 
empowers children to take control of their own learning (AIED by 
another name). Meanwhile, AI is being introduced into some mainstream 
schools as a curriculum in its own right,’ is being developed to improve 
online tutoring,’ and is being researched as a way of enhancing teacher 


training." In short, the application of AI in educational contexts is 


growing exponentially,’ such that by 2024 it is predicted to become a 


market worth almost $6 billion.'®” 

While we may have some limited knowledge or experience of 
mainstream AI, either from the media or in our daily lives, for many the 
use of AI in education remains a mystery. A multitude of yet-to-be- 
answered questions spring to mind. How exactly can AI work in 
classrooms, and what can be achieved? With AI requiring so much data, 
how is student privacy maintained? What will be APs long-term effects 


on teacher roles? Are the proponents of AIED promising more than can 


158 Woolf, B. (1988). “Intelligent tutoring systems: A survey.’ In Exploring Artificial Intelligence: 1—43; Cumming, 
G., and McDougall, A. (2000). “Mainstreaming AIED into education?” International Journal of Artificial 
Intelligence in Education 11: 197—207; du Boulay, B. (2016). “Artificial intelligence as an effective classroom 
assistant.” IEEE Intelligent Systems 31 (6): 76-81. https://doi.org/10.1109/MIS.2016.93 

153 https://www.linkedin.com/pulse/tech-giants-quietly-invest-adaptive-learning-system-rd-drew-carson 

160 http://www.knewton.com 

‘61 http://www.carnegielearning.com 

16 https://learning.xprize.org 

16 http://www.gettingsmart.com/201 8/07/coming-this-fall-to-montour-school-district-americas-first-public- 
school-ai-program 

‘6+ Following a $300m investment, the Chinese online tutoring company Yuanfudao has set up a research 
institute for artificial intelligence, which aims to train its homework app to be smarter. 
https://techcrunch.com/2018/12/26/yuanfudao-raises-300-million/ 

165 ©’Connell, S. (2018). “New Project Aims to Use Artificial Intelligence to Enhance Teacher Training.” 
Center for Digital Education. http://www.govtech.com/education/higher-ed/New-Project-Aims-to-Use- 
Artificial-Intelligence-to-Enhance-Teacher-Training. html 

16 https://www.eschoolnews.com/2017/05/22/brace-ai-set-explode-next-4-years 


167 https://www.gminsights.com/industry-analysis /artificial-intelligence-ai-in-education-market 
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be delivered? What is the impact of AI on student agency and outcomes? 
And what are the social and ethical consequences? 

However, we begin with a tentative response to an ostensibly simpler 
question: what does AIED actually look like? 


AI in Education 


As a brief review of AIED conference and journal papers will confirm, 
AIED includes everything from AI-driven, step-by-step personalized 
instructional and dialogue systems, through AI-supported exploratory 
learning, the analysis of student writing, intelligent agents in game-based 
environments, and student-support chatbots, to Al-facilitated 
student/tutor matching that puts students firmly in control of their own 
learning. It also includes students interacting one-to-one with computers, 
whole-school approaches, students using mobile phones outside the 
classroom, and much more besides. In addition, AIED can also shine a 
light on learning and educational practices. 

The field of AIED is both derivative and innovative. On the one 
hand, it brings theories and methodologies from related fields such as AI, 
cognitive science, and education. On the other hand, it generates its own 
larger research issues and questions: What is the nature of knowledge, 
and how is it represented? How can an individual student be helped to 
learn? Which styles of teaching interaction are effective, and when should 
they be used? What misconceptions do learners have?"® 

While ATED tools necessarily instantiate specific learning theories 


(such as Gagné’s “instructionalism’’'” 


ot Vygotsky’s “zone of proximal 
development”), some AIED researchers question the assumptions 
behind those theories, applying AI and data analysis techniques to try to 


open the “black box of learning,’””"”" In other words, AIED effectively 


18 Woolf, B.P. (2010). Building Intelligent Interactive Tutors: Stndent-Centered S. trategies for Revolutionizing e-Learning. 
Morgan Kaufmann, 11. 

' Gagné, R.M. (1985). Conditions of Learning and Theory of Instruction, 4th Revised Edition. Wadsworth Publishing 
Co Inc. 

1 Vygotsky, L. S. (1978). Mind in Society: Development of Higher Psychological Processes. Harvard University Press. 
171 Luckin, R., et al. Intelligence Unleashed. An Argument for Al in Education. 
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involves two main complementary strands: developing Al-based tools to 
support learning, and using these tools to help understand learning (how 
learning happens and other questions that have long been investigated by 
the learning sciences, and which might be applied in classrooms whether 
or not AI is being used). For example, by modeling how students go 
about solving an arithmetic problem and identifying misconceptions that 
might have been previously unknown to educators, researchers and 
teachers can begin to understand much more about the process of 
learning itself, which might then be applied to mainstream classroom 
practices. 

In fact, new AIED applications and approaches, addressing old and 
newly identified problems, are being researched and released all the 
time—such that, what AIED looks like and can do is still emerging. 
Accordingly, here we adopt an alternative approach. Rather than trying to 
define AIED, within some relatively easy-to-identify broad areas we will 
discuss a wide range of AIED examples—existing AIED tools and 
AIED tools that might be available in the not-too-distant futute. 

What this book does not include, but what might nonetheless have a 
major impact on education, is the use of AI to support school and 
university administrative functions such as class timetabling, staff 
scheduling, facilities management, finances, cybersecurity, safety and 
security. Instead, we focus on the use of AI to support learning, what 
might be called the academic (or system-facing) functions of AIED. 

However, before doing so, it will be helpful to have at least a working 
understanding of AI itself.’ That is where we go now, before returning 
to look in more detail at how AI works in educational contexts. We 
conclude by considering the various challenges, pragmatic and ethical, 
from the perspectives of AIED researchers and developers, as well as of 


educators, students, funders and policy-makers. 


172 Tt has been argued that “Artificial Intelligence should be accessible to all of us, even without a math 
background.” https://www.youtube.com/watch?v=LqjP7O9SxOM&list=PLtmWHNX-gukLQIMvtR] 19s7- 
8MrnRV6h6 
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The Background of AI 


AI is one of those aspects of modern life about which most of us have 
some awareness, and yet recognize we have little knowledge.'” In fact, 
for many AI is synonymous with humanoid robots,'”* which might be 
because news about AI is almost always illustrated with a picture of a 
robot or a digital brain. However, while robotics (embodied AI, that can 
move and physically interact with the world) is a core area of AI research, 
AI is being applied in many different ways and different contexts. 
Meanwhile, the dystopian images of futuristic robots remain firmly in the 
realm of science fiction (which is why for the most part we leave robotics 
well alone). In the next few pages, we provide a brief background to 
artificial intelligence; interested readers will find mote information about 
the origins and development of AI and its various techniques in appendix 
2; 

However, first, we should acknowledge that the very name artificial 
intelligence is sometimes seen as unhelpful. Instead, some researchers 
prefer augmented intelligence, which retains the human brain as the 
source of intelligence, and positions the computer and its programs as a 
sophisticated tool with which humans might enhance or augment our 
intellectual capabilities. In this approach, computers are employed to do 
what humans find more difficult (such as finding patterns in huge 
amounts of data). The debate contrasting augmented and artificial will 
inevitably run and run, with artificial intelligence winning at least on 
popular usage even if augmented intelligence is more accurate or useful. 
Accordingly, hereafter we will take the ultimate pragmatic approach and 
refer almost exclusively to AI, leaving the reader to decide for themselves 
what the A in AI represents. 


13 E.g., “What is artificial intelligence?” https://www.brookings.edu/research /what-is-attificial-intelligence 
17 The question “When you think of AI, what is the first thing that comes into your head?” has been posed in 
numerous lectures and surveys about AI in education. The evidence is anecdotal, but overwhelmingly 


participants answer: robots. 
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A 1956 workshop held at Dartmouth College, a US Ivy League 
research university, is widely considered to be AI’s foundational event.” 
It was where what is thought to be the first AI program, the Logic 
Theorist, was presented and discussed. Over the following decades, AI 
developed in fits and starts with periods of rapid progress (for example, 
tule-based expert systems) interspaced with periods known as AI winters 
where confidence and funding all but evaporated. 

In recent decades, thanks to three key developments (the advent of 
faster computer processors, the availability of large amounts of big data, 
and advances in computational approaches) AI has entered a period of 
renaissance—AI has now become an integral, pervasive, and inescapable, 
although often hidden, part of our daily lives. In fact, paradoxically, the 
mote that it is integrated, the less we tend to think of it as AI. 


A lot of cutting edge AI has filtered into general 
applications, often without being called AI because once 
something becomes useful enough and common enough 
it’s not labeled AI anymore.!76 
Instead, AI is often known as an advanced computer program (such 
as email spam filtering), "a personal assistant (such as Cortana), ° a 
recommendation system (such as in Netflix),'” or perhaps a language- 
learning app (such as Duolingo).'*° Having said that, recent voice- 
activated smart speakers, such as Google Home" and Amazon Echo,” 
have made AI more visible in out living rooms. In fact, many recent 
developments in AI are both groundbreaking and in many ways 
transformative. Relatively recent AI computational approaches, such as 


" Crevier, D. (1993). AI. The Tumutlinous History of the Search for Artificial Intelligence. Basic Books. 

176 http://edition.cnn.com/2006/TECH/science/07/24/ai.bostrom/index.html (Professor Nick Bostrom, 
director of the Future of Humanity Institute, University of Oxford). 

17 E.g., https://www.mailwasher.net uses Bayesian techniques to learn which emails are spam and which are 
not. 

178 https://www.microsoft.com/en-us/cortana 

1 https://help.netflix.com/en/node/9898 

18 https://www.duolingo.com 

181 https://store.google.com/gb/product/google_home 

182 https://www.amazon.com/b/?ie=UTF8&node=9818047011 
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machine learning (supervised, unsupervised, and reinforcement learning), 
neural networks (including deep learning), and evolutionary algorithms 
have all been used in a diverse range of applications (interested readers 
will find more information about these techniques in appendix 2). 

For example, recent advances in face recognition (ensuring that faces 
in smattphone photographs are always in sharp focus, and identifying 
travelers at e-passport gates) are thanks to the application of neural 
networks and machine learning. Google researchers presented a brain- 
inspired AI neural network with 10 million randomly selected video 
thumbnails from YouTube. By using deep-learning techniques, and 
despite not being told how to recognize anything in particular, this 
machine learning system soon learned how to detect human faces in 
photographs. Two years later, Facebook introduced a nine-layer deep AI 
neural network, involving more than 120 million parameters, to zdentify 
(not just detect) faces in timeline photographs.’ It was trained on a 
dataset of four million images of faces that had previously been labeled 
by humans (the Facebook users, who had been happily labeling their 
friends in uploaded photographs over several years). 

Another area that has seen much AI development in recent years is 
autonomous vehicles, with neural networks being used to enable cars, 
trucks, and taxis to drive without human intervention. A complex rig of 
cameras and sensors collate massive amounts of real-time data (the road’s 
edges and markings, road signs and traffic lights, other vehicles including 
bicycles, other potential obstacles, and pedestrians), while a neural 
network-driven intelligent agent, drawing on massive of computing 
power, controls the car’s steering, acceleration, and braking. A probably 
less well-known use of AI is in journalism. News organizations around 
the world are developing AI technologies to support their news gathering 
and news reporting. For example, AI agents continually monitor global 


183 https://www.nytimes.com/2012/06/26/technology/in-a-big-network-of-computers-evidence-of-machine- 
learning.html?_r=1 

'® Facebook introduced a nine-layer deep AI neural network, involving more than 120 million parameters, to 
identify (not just detect) faces in timeline photographs. It was trained on a dataset of four million images. 
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news outlets and use semantic analysis to automatically extract key 
information that is made available to the journalists to write their 
stories." There are even some AI technologies that go one step further 
and automatically write the stories themselves.'*° 

Another application of AI is in law, where e-Discovery tools are being 
used by lawyers to help process the huge amounts of documentation that 
need to be reviewed as potential evidence in civil or criminal legal 
cases.'*’ One technique involves a machine learning analysis of a sample 
of documents that have been reviewed and labeled by an expert. The 
outcomes enable the AI to then identify which of the remaining 
documents need to be prioritized for in-depth review. A final brief 
example is the use of AI in medical diagnoses. For example, AI 
techniques are used by radiologists to help them identify anomalies in 
medical images more quickly while making fewer mistakes.'** One system 
looks for irregularities in X-ray images. If, for example, it finds nodules 
on an image of a pair of lungs, it sends it to a pulmonary radiologist for 


further checks. 


AI Techniques and Terminology 


While it is relatively straightforward to understand what the applications 
of AI outlined in the previous section are doing, understanding how they 
ate doing it can require some highly technical knowledge—exacerbated 
by the fact that any one AI application might draw on several different 
AI techniques. This is one reason why many people involved in AI have 
advanced degrees in mathematics or physics (although AI is increasingly 
being offered as a service: for example, Amazon’s Machine Learning on 
AWS,” Google’s TensorFlow,'” IBM’s Watson,” and Microsoft’s 


18 E.g., http://bbcnewslabs.co.uk/projects/juicer 

1% E.g., https://narrativescience.com/Products/Our-Products/Quill 

187 E g, https://talkingtech.cliffordchance.com/en/emerging-technologies/artificial-intelligence/ai-and-the- 
future-for-legal-services.html 

188 Hosny, A., et al. (2018). “Artificial intelligence in radiology.” Nature Reviews Cancer 18 (8): 500-510. 
https://doi.org/10.1038/s41568-018-0016-5 

189 https://aws.amazon.com/machine-learning 


1% https://www.tensorflow.org 
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Azure). Nonetheless, because some AI techniques have already been 
repeatedly mentioned, and because they play important roles in AIED 
and so will be mentioned again, some key and closely interlinked AI 
techniques and terminologies will next be introduced.'™ At times (despite 
our best efforts) our discussion will be somewhat technical; so please feel 
free to jump to the next section, to move directly onto our discussion of 
the application of AI in education (which, after all, is the reason we ate all 
here). 


Algorithms 


Algorithms are at the core of AI, such that the history of AI might be 
thought of as the history of the development of increasingly 
sophisticated and increasingly efficient (or elegant) algorithms. Probably 
the most famous algorithm of recent times is PageRank,'”* which was 
developed in 1996 by the founders of Google while they were students at 
Stanford University. It ranks the relative importance of a website, by 
counting the number of external links to the website’s pages, to 
determine where the website appeared in a Google search. In fact, all 
computer programs are algorithms. They comprise hundreds if not 
thousands of lines of code, representing sets of mathematical instructions 
that the computer follows in order to solve problems (compute a 
numerical calculation, grammar-check an essay, process an image, of 
explain patterns that we see in nature). All that makes AI algorithms 
distinct from other computer programs is that they involve some specific 
approaches and, as we have noted, they are applied to areas we might 
think of as essentially human—such as visual petception, speech 
recognition, decision-making and learning. 


ee ee Ee 


11 https://www.ibm.com/watson 

19 https://azure.microsoft.com 

193 Readers wishing to learn more about AI techniques might be interested in Russell, S. and Norvig, P. (2016). 
Artificial Intelligence: A Modern Approach, 3rd Edition. Pearson; and Domingos, P. (2017). The Master Algorithm: 
How the Quest for the Ultimate Learning Machine Will Remake Our World. Penguin. 

1% PageRank of Site = X [PageRank of inbound link/Number of links on that page]. 

15 Turing, A. (1952). “The chemical basis of morphogenesis.” Philosophical Transactions of the Royal S ociety 237 
(641): 37-72. 
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Machine Learning 


Much early, rule-based AI involves writing in advance the steps that the 
computer will take to complete a task, rules that will be followed exactly. 
Machine learning, on the other hand, is about getting computers to act 
without being given every step in advance. Instead of the algorithms 
being programmed exactly what to do, broadly speaking they have the 
ability to learn what to do. This is not to suggest that machine learning 
does not require large amounts of programming, because it does. But 
rather that, instead of direct commands leading to direct outputs, 
machine learning involves large amounts of input data to predict novel 
outcomes. 

Machine learning algorithms analyze the data to identify patterns and 
to build a model which is then used to predict future values (for example, 
by identifying patterns in historical stocks data, AI predicts future stock 
movements; by identifying patterns in photographs of named people, it 
predicts who is shown in other photographs; and by identifying patterns 
in medical symptoms, it predicts a specific diagnosis). In other words, 
machine learning may be considered a three-step process (analyze data, 
build a model, undertake an action) that is continuously iterated (the 
outcomes of the action generate new data, which in turn amends the 
model, which in turn causes a new action). It is in this sense that the 
machine is learning. 

Many recent applications (including natural language processing, self- 
driving cars, and the Google DeepMind AlphaGo program that beat the 
world’s number one player of Go)'”* have all been made possible thanks 
to machine learning. In fact, machine learning is so widespread today 
that, for some commentators, AI and machine learning have become 
synonymous—whereas machine learning is more properly a sub-field of 
AI. What is true, however, is that the renaissance and exponential growth 


of AI over the last decade, has come about because of significant 


196 https://www.theguardian.com/technology/20 16/mar/1 5/googles-alphago-seals-4-1-victory-over- 


grandmaster-lee-sedol 
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advances in machine learning (based on, as we have noted, faster 
computer processors, the availability of large amounts of big data, and 
new computational approaches).'”” 

There are three main categories of machine learning: supervised, 
unsupervised, and reinforcement learning. 


Supervised Learning 


Most practical machine learning involves supervised learning. The AI is 
first provided large amounts of data for which the Output is already 
known—in other words, data that has already been labeled. For example, 
the AI might be given many thousands of photographs of streets in 
which the numerous visible objects (bicycles, toad signs, pedestrians, etc.) 
have already been identified and labeled by humans. The supervised 
learning algorithm aims to identify the function that links the data to the 
labels, ftom which it builds a model that can be applied to new similar 
data. This is broadly speaking the approach, mentioned earlier, used by 
Facebook to identify people in photographs, which used millions of 
photographs submitted and labeled by Facebook users to identify and 
label automatically the same people in new photographs. 


Unsupervised Learning” 


In unsupervised learning, the AI is provided with even larger amounts of 
data, but this time data that has not been categorized or classified, that is 
to say data that is not labeled. By analyzing this unlabeled data, 
unsupervised learning algorithms aim to uncover hidden patterns in the 


underlying structure of the data, clusters of data that can be used to . 


classify new data (this is broadly the approach, mentioned earlier, used by 
Google to detect faces in photographs). Example applications of 
unsupervised learning include dividing online shoppers into groups so 


'7 Interestingly, the origins of machine learning can be traced back to at least 1959, with the publication of 
“Some Studies in Machine Learning Using the Game of Checkers” by an IBM researcher. 

8 A comprehensive list of the algorithms available on one of the leading “AI as a service” platforms, 
Microsoft Azure, is available at http://download.microsoft.com/download/A/6/1 /A613E11E-8F9C-424A- 
B99D-65344785C288/mictosoft-machine-leatning-algorithm-cheat-sheet-v6.pdf 
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they can be served tightly targeted advertisements;'” identifying different 
letters and numbers from examples of handwriting; and distinguishing 


between legitimate and fraudulent financial transactions. 


Reinforcement Learning 


In some senses, reinforcement learning is the most powerful of the 
machine learning categories. In both supervised and unsupervised 
learning, the model derived from the data is fixed, and if the data changes 
the analysis has to be undertaken again (in other words, the algorithm is 
run once more). However, reinforcement learning involves continuously 
improving the model based on feedback—in other words, this is machine 
learning in the sense that the learning is ongoing. The AI is provided with 
some initial data from which it derives its model, which is evaluated, 
assessed as correct or incorrect, and rewarded or punished accordingly 
(to use a computer game metaphor, its score is increased or reduced). 
The AI uses this positive or negative reinforcement to update its model 
and then it tries again, thus developing iteratively (learning and evolving) 
over time. For example, if an autonomous cat avoids a collision, the 
model that enabled it to do so is rewarded (reinforced), enhancing its 


ability to avoid collisions in the future. 


Artificial Neural Networks 


An artificial neural network is an AI algorithm that is based on the 
structure and functions of biological neural networks (i.e. animal brains), 
that might be applied in advanced supervised, unsupervised, or 
reinforcement learning. Our brains are made up of billions of individual 
neurons, each of which is connected to as many as a thousand other 
neurons, giving trillions of connections. Memory is thought to emerge 
from complex combinations of these connections across the brain, while 
learning is thought to involve the strengthening of those connections. 


® In a now-infamous story, the US retailer Target automatically identified a teenager as being pregnant, 
before she had told anyone, just by her store purchases, and some unsupervised learning. 
https://www.forbes.com/ sites /kashmirhill/201 2/02/16/how-target-figured-out-a-teen-girl-was-pregnant- 
before-her-father-did/#31650c296668 
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Although artificial neural networks have been trained to do some 
incredible things (such as identifying faces in moving crowds of people), 
they remain primitive in comparison to higher-order animal brains. 
Unlike for example the human brain’s billions of neurons, they usually 
involve only a few thousand neurons (in some exceptional cases, a few 
million). 

As illustrated in the following figure, artificial neural networks each 
comprise three types of layers: an input layer that takes stimuli from the 
environment, in the form of millions of data points, perhaps pixels from 
images; at least one, but often many mote, hidden intermediary layers 
that together undertake the computation; and an output layer that 
delivers the result. During the machine learning process, weightings given 
to the connections are adjusted in a process of reinforcement learning, 
which allows the artificial neural network subsequently to compute 
outputs for new stimuli. 


midden layer | hidden layer 2 


A representation of a typical, simple artificial neural network, 
with two hidden layers. 


The hidden layers are the key to the power of artificial neural 
networks, but they also bring an important problem. It isn’t possible (or 
at the very least it isn’t easy) to interrogate an artificial neural netwotk to 
find out how it came up with its solution—for example, how did it 
identify a particular person in a photograph? In other words, artificial 
neural networks can lead to decision making for which the rationale is 
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. . ` + 200 
hidden and unknowable, or un-inspectable, and possibly unjust," a 
wa . . . 201 
critical issue that is the subject of much research. 
Finally, the impressive results of neural networks and other machine 


learning technologies should not beguile us: 


A neural network of today no more “learns” or “reasons” 
about the world than a linear regression of the past. They 
merely induce patterns through statistics. Those patterns 
may be opaquer, more mediated and more automatic than 
historical approaches and capable of representing more 
complex statistical phenomena, but they are still merely 
mathematical incarnations, not intelligent entities, no 
matter how spectacular their results.292 


200 O’Neil, C. (2017). Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy. 


Penguin. 


21 Morcos, A.S., et al. (2018). “On the importance of single directions for generalization.” ArXiv:1803.06959. 


http://arxiv.org/abs/1803.06959 
20 hetps://www.forbes.com/sites/kalevleetaru/2018/12/15/does-ai-truly-learn-and-why-we-need-to-stop- 
overhyping-deep-learning/#edd206168c02 
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How AI Works in Education 


Having established a working understanding of AI, we will now look in 
more detail at how AI works in educational contexts, beginning with a 
brief history. However, what will not be discussed but what might 
nonetheless have a major impact on education is the use of AI to support 
school administration (system-facing AI that addresses things like class 
timetabling, staff scheduling, facilities management, finances, 
cybersecurity, safety and security). Our focus is the use of AI to 


support learning (student- and teacher-facing AT). 


A Brief History of AI in Education 


Precursors of the application of AI in education can be found in the 
work of the psychologists Sidney Pressey, who was a professor at Ohio 
State University in the 1920s, and B. F. Skinner, known as the father of 
behaviorism, who was a professor at Harvard University from 1948 until 
his retirement in 1974, For Pressey, the challenge was to leverage the 
potential of multiple-choice tests to consolidate student learning as well 
as to evaluate it. Drawing on Edward Thorndike’s law of effect,* he 
argued that, for tests to support learning, immediate feedback was 
essential—which is not usually possible when tests are marked by hand. 
However, a mechanical: approach could ensure that no learning 
oppottunities were missed. 


Devices which at once inform a student about the 
correctness of his answer to a question, and then lead him 
to the right answer, clearly do more than test him; they 
also teach him.205 


203 Readers who are interested in the use of AI technologies to support administrative functions might like to 
read about Ofsted, the UK’s school inspection service. Ofsted’s use of “artificial-intelligence algorithm to 
predict which schools are 'less than good',” https://www.tes.com/news/ofsted-use-artificial-intelligence- 
algorithm-predict-which-schools-are-less-good 

2 Thorndike. E.L. (1927) “The Law of Effect.” The American Journal of Psychology 39 (1/4): 212-22. 
https://doi.org/10.2307/1415413 

205 Pressey, S.L. (1950). “Development and appraisal of devices providing immediate automatic scoring of 
objective tests and concomitant self-instruction.” Journal of Psychology 30: 417-447. 
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Pressey made various versions of his machine (and made several 
unsuccessful attempts to commetcialize his idea), the most sophisticated 
being based on a mechanical typewriter. Inside this device was a rotating 
drum around which was wrapped a card printed with a list of questions 
and hole-punched (much like the perforated rolls used in self-playing 
pianos) to represent the correct answers. Meanwhile, the casing featured 
a small window, which showed the number of the current question, and 
five typewriter keys, one for each possible answer. As the student worked 
through a printed sheet of questions and answers, they would press one 
of the keys on the device to select their answer for each question. The 
machine was configured so that the student immediately knew whether 
they had made the right choice, and it prevented them from moving onto 
the next question until they had. 

Interestingly, Pressey was also one of the first to make the case that, in 
addition to supporting learning, a teaching machine could make a 
teachet’s life easier and more fulfilling—by relieving them of one of their 
least interesting tasks (marking tests) and giving them more time to 


engage with their students. 


Lift from [the teacher's] shoulders as much as possible of 
this burden and make her [sic] free for those inspirational 
and thought-stimulating activities which are, presumably, 
the real function of the teacher. 

Pressey’s approach was later extended by Skinner, who argued that the 
techniques he pioneered for training rats and pigeons (in operant 
conditioning chambers now known as Skinner Boxes) might be adapted 
for teaching people. Skinnet’s teaching machine, which he devised in 
1958, was a wooden box with a windowed lid. Questions written on 
paper disks appeared in one window, and the student wrote a response 
on a roll of paper accessible through a second window (for later marking 


by a teacher). Advancing the mechanism automatically covered the 


206 Pressey, S.L. (1926). “A simple device for teaching, testing, and research in learning.” School and Society 23: 
374. 
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student’s answer, so that it could not be changed, and simultaneously 
revealed the correct answer. In this way, Skinnet’s teaching machine 
provided automatic, immediate reinforcement. Students were required to 
compose their own answers, rather than choose from a limited selection 
(as with Pressey’s multiple-choice questions), because Skinner found that 
learning is more effectively reinforced by recalling a correct response 
than by simply recognizing it. This approach also gave the student the 
Opportunity to compare their answer with the given model answer, which 
if properly designed by the teacher and actively undertaken by the 
student could also contribute to learning. 

Skinner argued that his teaching machine in effect acted like a 
personal tutor, foreshadowing AIED’s intelligent tutoring systems. 


The machine itself, of course, does not teach ... but the 
effect upon each student is surprisingly like that of a 
private tutor... (i) There is a constant interchange between 
program and student.... (ii) Like a good tutor, the machine 
insists that a given point be thoroughly understood ... 
before the student moves on.... (iii) Like a good tutor, the 
machine presents just that material for which the student is 
ready... (iv) Like a skillful tutor, the machine helps the 
student to come up with the right answer.... (v) Lastly, of 
course, the machine, like the private tutor, reinforces the 
student for every correct response, using this immediate 
feedback ... to shape his behavior most efficiently.207 


Skinnet’s teaching machine might be thought to have also 
foreshadowed something else later taken up by AI in education 
researchers, dividing automated teaching into separate components (in 
Skinnet’s case, distinguishing between the subject content, which was 
pre-programmed into the machine, and the students achievements, 
whether or not they answered a question correctly). However, although 
in a sense Skinners teaching machine was responsive to individual 
students, it could not be considered adaptive. That is to say, it did not 


adapt either the questions, or the order in which they were presented, 


7 Skinner, B.F. (1958). “Teaching machines.” Science 128 (3330): 969-77. 
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accotding to the achievements or needs of the individual students. 
Instead, question delivery was pre-scripted. While a student could 
proceed at their own pace, they went through the same list of questions 


as every other student and in the same order. 


Adaptive Learning 


Also working in the 1950s, Norman Crowder, who was interested in 
communication rather than psychology, devised a paper-based alternative 
to the early teaching machines, known as intrinsic or branching 
programmed instruction.” In Crowder’s system (which he developed for 
training U.S. Air Force engineers to find malfunctions in electronic 
equipment), the user is presented with a short page of information 
followed by a multiple-choice question, with each possible answer 
directing the student to a new page. If the correct answer was chosen, the 
new page would present new information, building upon that which was 
correctly understood; if an incorrect answer was chosen, the new page 
would contain feedback designed to help the student understand the 
cause of their error, based on what the student had chosen. The system 
might also branch through one or two additional pages of corrective 
materials before returning the student back to the main pages. In short, 
Crowdet’s system adapted the pathway through the teaching materials 
accotding to the individual student’s developing knowledge, such that 
each student might see quite different sets of pages. 

However, a British polymath, Gordon Pask, probably developed the 
first truly adaptive teaching machine in the early 1950s. Known as SAKI 
(the self-adaptive keyboard instructor), it was designed for trainee 
keyboard operators learning how to use a device that punched holes in 
cards for data processing.” What distinguished SAKI from the other 


eatly teaching machines was that the task presented to a learner was 


208 Crowder, N.C. (1960). “Automatic tutoring by means of intrinsic programming.” In Teaching Machines and 
Programmed Learning: A Source Book. Vol. 116. Lumsdaine, A.A., and Glaser, R. (eds.) American Psychological 
Association, 286-298. 

209 Pask, G. (1982). “SAKI: Twenty-five years of adaptive training into the microprocessor era.” International 
Journal of Man-Machine Studies 17 (1): 69-74. https://doi.org/10.1016/S0020-7373(82)80009-6 
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adapted to the learner’s individual performance, which was represented 
in a continuously changing probabilistic student model. 

When you interact with the system, learning which keys represent 
which numbers: 


the machine is measuring your responses, and building its 
own probabilistic model of your learning process. That 
“7,” for instance, you now go to straight away. But the 
“3,” for some obscure reason, always seems to elude you. 
The machine has detected this, and has built the facts into 
its model. And now, the outcome is being fed back to you. 
Numbers with which you have difficulty come up with 
increasing frequency in the otherwise random presentation 
of digits. They come up more slowly, too, as if to say: 
“Now take your time.” The numbers you find easy, on the 
contrary, come up much faster: the speed with which each 
number is thrown at you is a function of the state of your 
learning.?10 


Computer-Aided Instruction 


SAKI went through many iterations, taking advantage of developments 
in computers and the new microprocessors, and was one of the first 
adaptive systems to be commercialized. However, over the following 
years, other than in the various iterations of SAKI, adaptive learning 
made few advances, and the focus shifted to what became known as 
computer-aided instruction (CAI) systems. The 1960s and 1970s saw 
many CAI systems being built, an early influential example being PLATO 
(programmed logic for automatic teaching operations), which was 
developed at the University of Illinois. PLATO involved students 
accessing standard teaching materials, some of which were interactive, on 
a central mainframe computer via remote terminals, with as many as a 
thousand students working at the same time. 

This system was also notable for being the first to introduce in an 
educational technology many tools and approaches still common today, 


such as user forums, email, instant messaging, remote screen-sharing, and 


21° Beer, S. (1960). Cybernetics and Management. The English Universities Press, 124. 
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multiplayer games. Around the same time, Stanford University and IBM 
developed a computer-aided instruction system that was made available 
via remote terminals to a few local elementary schools. This system 
involved a linear presentation of teaching materials, for mathematics and 
language arts, together with drill and practice activities. A third 
prominent example was TICCIT (time-shared interactive computer- 
controlled information television), developed by Brigham Young 
University, which was used to teach freshman-level mathematics, 
chemistry, physics, English, and various language courses. Each subject 
area was broken down into topics and learning objectives, which in turn 
were represented as screens of information. TICCIT then provided a 
predetermined sequence, although learners could also use the keyboard 
to navigate through the screens in any order that they found helpful. 
Although in other ways successful, during the 1960s and 1970s only 
very few of these CAI systems were widely adopted, mainly due to the 
cost and accessibility of the university mainframes that were needed to 
host the software. The arrival of personal computers in the 1980s 
changed everything, with the number of CAI programs quickly 
mushrooming. Very soon, CAI programs addressing every aspect of 
learning were being widely used in schools, universities and family 
homes. Nonetheless, of particular relevance for our present purposes, 
almost all of these systems were severely hampered by the same flaw—a 
lack of adaptivity. The sequence of topics, the information provided, and 
the system’s response to student actions, were predefined and the same 
for each student, ignoring the individual students successes, 


misconceptions, and interests, and thus inhibiting their potential learning. 


AI and CAI 


At around the same time that mainstream CAI was developing, other 


san: 212 : 
researchers, such as John Self" and William Clancey,”” were wondering 


211 Self, J.A. (1974). “Student models in computer-aided instruction.” International Journal of Man-Machine Studies 
6 (2): 261-76. https://doi.org/10.1016/S0020-7373(74)80005-2 
212 Clancey, W.J. (1983). “GUIDON.” Journal of Computer-Based Instruction 10 (1-2 ): 8-15. 
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how CAI could become adaptive to the needs of individual students, and 
whether the application of techniques from AI might be useful. The first 
implementation of AI techniques in computer-aided instruction is usually 
credited to Jaime Carbonell, who, in his 1970 PhD thesis, introduced a 
system called SCHOLAR. This novel system was designed to enable a 
student to review their knowledge of the geography of South America, by 
means of a dialogue using a subset of English. SCHOLAR was 
qualitatively different from standard CAI because, unique for the time, it 
was capable of generating individual responses to student statements by 
drawing from a semantic network (a way of representing knowledge, in 
this case knowledge of geography, in which concepts are linked 
according to their semantic relationships). 


In [a] conventional CAI, the database consists of many 
“frames” of specific pieces of text, questions, and 
anticipated answers entered in advance by the teacher. By 
contrast, [SCHOLAR] is based on the utilization of an 
information network of facts, concepts, and procedures; it 
can generate text, questions, and corresponding answers. 
Because [it] can also utilize its information network to 
answer questions formulated by the student, a mixed- 
initiative dialogue between student and computer is 
possible with questions and answers from both sides.213 


A SCHOLAR dialogue?"* might look something like this: 


SCHOLAR: Use one of the following: Sucre, French, 
Argentina, Spanish, to answer the question: What is the 
language in Chile? 

SCHOLAR: You are taking too much time... 

Student: *Spanich* 

SCHOLAR: You should have written “Spanish.” Very 
good. 


Although by today’s standards the dialogue (and especially its 
pedagogy) appears primitive, SCHOLAR is usually considered to be the 


213 Carbonell, J. R. (1970). “AI in CAI: An Artificial-Intelligence Approach to Computer-Assisted Instruction.” 
IEEE Transactions on Man-Machine Systems 11 (4): 190-202. https://doi.org/10.1109/TMMS.1970.299942 
214 Carbonell, “AI in CAT,” 192. 
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first example of what came to be known as Intelligent Tutoring Systems, 


to which we turn next. 
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Applications of AI in Education 
Intelligent Tutoring Systems 


Under the best learning conditions we can devise 
(tutoring), the average student is two sigma above the 
average control student taught under conventional group 
methods of instruction. The tutoring process demonstrates 
that most of the students do have the potential to reach 
this high level of learning. I believe an important task of 
research and instruction is to seek ways of accomplishing 
this under more practical and realistic conditions than the 


one-to-one tutoring, which is too costly for most societies 

to bear on a large scale. This is the 2 sigma problem.215 
So-called intelligent tutoring systems (ITS) are among the most common 
applications of AI in education (in any case, as we have seen, they have | 
probably been around the longest). Generally speaking, ITS provide step- | 
by-step tutorials, individualized for each student, through topics in well- | 
defined structured subjects such as mathematics or physics.” Drawing 
on expert knowledge about the subject and about pedagogy, and in 
response to individual students misconceptions and successes, the 
system determines an optimal step-by-step pathway through the learning 
materials and activities. As the student proceeds, the system automatically | 
adjusts the level of difficulty and provides hints or guidance, all of which 
aim to ensure that the student is able to leatn the given topic effectively. 

TTS come in many shapes, although typically they involve several AI 

models, an approach that we will unpack here. As we saw in our earlier 
discussion of AI technologies, AI models are highly simplified 
computational representations (in semantic networks, as used by 


ee 


715 Bloom, Benjamin S. (1 984). ‘The 2 Sigma problem: The search for methods of group instruction as 
effective as one-to-one tutoring.” Educational Researcher 13 (6): 4. Note, however, that according to VanLehn, 
“human tutors are 0.79 sigma more effective than no tutoring and not the 2.0 sigma found in the Bloom 
(1984) studies” VanLehn, K, (2011,) “The relative effectiveness of human tutoring, intelligent tutoring 
systems, and other tutoring systems.” Educational Psychologist 46 (4): 209, 
https://doi.org/10.1080/00461520.2011.611369 

716 Alkhatlan, A. and Kalita, J. (2018). “Intelligent tutoring systems: A comprehensive historical survey with 
recent developments.” ArXiv:1812.09628. http://arxiv.org/abs/1812.09628 
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SCHOLAR, in ontologies,” or in knowledge graphs)"* of specific 
knowledge about the real world (just like a model car is a simplified 
representation of a real car). The models used by ITS represent 
knowledge specific to teaching and learning. Typically, knowledge about 
the topic to be learned is represented in what is known as a domain 
model, knowledge about effective approaches to teaching is represented 
in a pedagogical model, and knowledge about the student is represented 
in a learner model.” The ITS algorithm draws on these three models in 
order to adapt a sequence of learning activities for each individual 
student. A fourth model found in some ITS is the open learner model, to 


which we will return later. 


The Domain Model 


A domain model represents knowledge about the subject that the ITS 
aims to help the students learn (much like the subject knowledge in a 
standard, non-educational, expert system). This might, for example, be 
knowledge about mathematical procedures, genetic inheritance, or the 
causes of World War I. In fact, over the years, mathematics for primary 
and secondary school students has dominated ITS. Mathematics, along 
with physics and computer science, are AIED’s low-hanging fruits 
because they are, at least at a basic level, well-structured and clearly 


defined. 


The Pedagogy Model 


The ITS pedagogy model represents knowledge about effective 
approaches to teaching and learning that have been elicited from teaching 


experts and from research in the learning sciences (although it should be 


217 Ontologies are a way of representing a domain’s concepts, data, components, entities and properties, and 
the relationships between them. Sowa, J.F. (1995). “Top-level ontological categories.” International Journal of 
Human-Computer Studies 43 (5): 669-85. https://doi.org/10.1006/ijhc.1995.1068 

218 Knowledge graphs are an alternative approach to ontologies, 

https://ontotext.com/knowledgehub/ fundamentals /what-is-a-knowledge-graph 

219 Luckin, R., et al. (2018). Intelligence Unleashed. An Argument for Al in Education, 18; Boulay, B. du., 
Poulovassilis, A., Holmes, W., and Mavrikis, M. (2018). “What does the research say about how artificial 
intelligence and big data can close the achievement gap?” 4. In Luckin, R. (ed.) (2018). Enhancing Learning and 
Teaching with Technology, 316-27. Institute of Education Press. 
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acknowledged that some ITS developers falsely assume that they have 
sufficient expertise in pedagogy). Pedagogical knowledge that has been 
represented in many ITS include knowledge of instructional 
approaches,” the zone of proximal development,” interleaved 
practice,” cognitive load,” and formative feedback.” For example, a 
pedagogical model that implements Vygotsky’s zone of proximal 
development will ensure that activities provided by the system to the 
student are neither too easy nor too challenging, one that implements 
individualized formative feedback will ensure that feedback is provided 


to the student whenever it might support the student’s learning. 
The Learner Mode/ 


As we have seen, some CAI effectively (although usually by another 
name) implemented versions of both domain and pedagogical models: 
knowledge of what was to be learned and knowledge of how to teach 
what was to be learned (for example, using linear or branching 
programmed instruction). However, what distinguishes Al-driven ITSs is 
that, as foreshadowed by Pask’s SAKI, they also include a learner model: 
“a representation of the hypothesized knowledge state of the student.” 
In fact, many ITS incorporate a wide tange of knowledge about the 
student—such as their interactions, material that has challenged the 


eS 


ze Hor example, many ITS set out to address student “learning styles” (Kumar, Amit, Ninni Singh, and Neelu 
Jyothi Ahuja. (2017). “Learning-styles based adaptive intelligent tutoring systems: Document analysis of 
armeler published between 2001 and 2016.” International Journal of Cognitive Research in Science, Engineering and 
Eayeaion 5 (2): 83-98. https://doi.org/10.5937/IJCRSEE1 702083K This construct that has been widely 
discredited, e.g., Kirschner, P.A. (2017). “Stop propagating the learning styles myth.” Computers & Education 
106: 166-171. https://doi.org/10.101 6/j.compedu.2016.12.006 
721 Bereiter, C. and Scardamalia, M. (1 989). “Intentional learning as a goal of instruction.” Knowing, Learning, 
and Instruction: Essays in Honor of Robert Glaser, 361-392. 
222 Vygotsky, Mind in S ociety, 86ff. 
~ Rohrer D., and Taylor, K. (2007). “The shuffling of mathematics problems improves learning.” Instructional 
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student, their misconceptions, and their emotional states while using the 
system—all of which can be used to inform what is being taught and 
how, together with what support needs to be provided and when. In fact, 
most ITSs go much further. The knowledge stored about the individual 
student is augmented with knowledge of all the students who have used 
the system so far, from which the system machine learns in order to 
predict which pedagogical approach and which domain knowledge is 
appropriate for any particular student at any specific stage of their 
learning. It is the learner model that enables ITS to be adaptive, and the 


machine learning that makes this adaptivity especially powerful. 


A Typical ITS Architecture 


The following figure shows how the domain, pedagogy, and learner 


models might be connected in a typical ITS. 
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In this exemplar architecture, the ITS algorithm draws on the domain, 
pedagogy, and learner models to determine what adaptive learning 
activity should be presented to the individual student and how it should 
be adapted to that student’s needs and capabilities. For example, in a 
mathematics ITS, the domain model might contain knowledge about 
quadratic equations, the pedagogy model might contain knowledge of an 
effective way to teach quadratic equations, and the learner model might 
contain knowledge about the student’s experience learning about 


quadratic equations in this ITS (for example, a misconception that they 
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exhibited, or the fact that this topic caused them some anxiety). The 
learner model will also contain knowledge of all students who have ever 
used this ITS to learn about quadratic equations. 

Drawing all of this together, the ITS algorithm will determine what 
adaptive learning activity to present to the student in the user interface— 
in other words, which specific aspect of quadratic equations to deliver 
(perhaps factorizing or completing the square) and what approach to use 
to best help the student learn about those aspects of quadratic equations 
(perhaps some instructional text, an image or video, or an interleaved 
practice activity), all of which is also dependent on the learner model (the 
knowledge of the individual’s and all students’ experience of learning 
quadratic equations in this ITS). 

While the student engages with the adaptive learning activity selected 
by the system, data capture involves the system capturing thousands of 
data points representing each individual interaction (what is clicked on 
the screen and what is typed, and possibly even how rapidly they move 
the mouse around the screen), together with the student’s achievements 
(which tasks they have answered correctly or partially) and any 
misconceptions that they have demonstrated. Some advanced ITS also 
capture other data such as the student’s speech, physiological responses, 
and an inference of their affective (emotional) state. 

The next step involves data analysis, in which all of the captured data 
is automatically examined, possibly using machine learning (or a Bayesian 
network, an AI technique that is introduced in appendix 2, both to 
provide the student with individualized formative feedback (to support 
their learning according to their individual needs) and to update the 
learner model (to inform the system’s decision of which adaptive learning 
activity to deliver next, and to contribute to the model of all students). 
The analysis might also, in some circumstances, update the pedagogy 
model (identifying which approaches have been shown to support 
student learning most or least effectively, in particular circumstances) and 
the domain model (perhaps with previously unknown misconceptions 
that have become apparent from the student interactions), 
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Over time, this ITS cycle—(a) drawing on the domain, pedagogy and 
learner models, (b) delivering adaptive learning activities, (c) data capture, 
(d) data analysis, and (e) updating the models—means that each 
individual student will experience their own unique personalized learning 
pathway through the available learning activities. If their interactions 
suggest that they find factorizing particularly challenging, perhaps they 
will spend more time engaging with multiple relevant activities; whereas if 
their interactions suggest otherwise, perhaps they made fewer errors 
along the way, they will work through fewer activities for this topic and 
will more quickly move onto another topic deemed to be more 
appropriate for their particular needs. 

Finally, as also shown in the preceding figure, a few ITS also feature a 
fourth model, known as an open learner model.” Open learner models 
aim to make visible or explicit, for both the students and their teachers to 
inspect the teaching and learning that has taken place and the decisions 
that have been taken by the system. The open learner model enables 
learners to monitor theit achievements and personal challenges, 
supporting their metacognition, and enables teachers to better 
understand each individual learners learning (their approach, any 
misconceptions and their learning trajectories) in the context of the 
whole class, as well as potentially informing the teacher’s professional 


development. 


Evaluating ITS 


Over the years there have been countless examples of ITS, many of 
which have been evaluated in schools or universities. Usually these 
evaluations have focused on learning gains, comparing one or other ITS 
with traditional teaching methods, such as whole class ot one-to-one 
teaching by a human teacher, or with CAI systems. In fact, as detailed by 


du Boulay and colleagues, there have also now been several meta- 


227 Dimitrova, V., Mccalla, G., and Bull, S. (2007). “Preface: Open learner models: Future research directions.” 
Special issue of the International Journal of Artificial Intelligence in Education, Part 2. 
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reviews” (i.e., review papers that aim to draw some general conclusions 
by combining and analyzing the trends in several individual evaluations). 
For example, one meta-analysis notes: “Developers of ITSs long ago set 
out to improve on the success of CAI tutoring and to match the success 
of human tutoring. Our results suggest that ITS developers have already 
met both of these goals.” However, pooling the outcomes of the 
several meta-studies suggests that ITSs have not yet quite achieved parity 
with one-to-one teachinge—when combined, the meta-teviews show an 
average small negative effect size of —0.19.”° On the other hand, for ITSs 
that are compared with whole-class teaching, the outcomes of the meta- 
reviews have been very positive. They show a weighted average effect 
size of 0.47”! —in educational intervention research, effect sizes above 
0.4 are thought to be “worth having,”?” 

As we mentioned at the start of this section, ITS tend to focus on 
well-defined domains such as mathematics or physics. However, it is also 
worth noting that over recent years ITS for ill-defined problems (such as 


legal argumentation, intercultural skills acquisition, and dispute 


a 


78 VanLehn, K. (2011), “The relative effectiveness of human tutoring, intelligent tutoring systems, and other 
tutoring systems.” Educational Psychologist 46 (4): 197-221. https://doi.org/10.1080/00461520.201 1.611369; 
Ma, W., et al. (2014). “Intelligent tutoring systems and learning outcomes: A meta-analysis.” Journal of 
Educational Psychology 106 (4): 901; Nesbit, J.C., et al. (2014) “How effective are intelligent tutoring systems in 
computer science education?” In 2014 IEEE 14th International Conference On Advanced Learning Technologies, 
http://iecexplore.icee.org/abstract/document/6901409/; Kulik, J.A. and Fletcher, J.D. (2015). “Effectiveness 
of intelligent tutoring systems a meta-analytic review.” Review of Educational Research, 
https://doi.org/10.3102/0034654315581420; Steenbergen-Hu, S. and Cooper, H. (2013). “A meta-analysis of 
the effectiveness of intelligent tutoring systems on K-12 students’ mathematical learning.” 
http://psycnet.apa.org/journals/edu/105/4/970/; Steenbergen-Hu, S. and Cooper, H. (2014). “A meta- 
analysis of the effectiveness of intelligent tutoring systems on college students’ academic learning.” 
http://psycnet.apa.org/journals/edu/106/2/331/ 
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Review of Educational Research, 0034654315581420, https://doi.org/10.3102/0034654315581420 

250 Although one meta-review did find that ITSs were “just as effective as adult, one-to-one tutoring”: 
VanLehn, “The relative effectiveness of human tutoring, intelligent tutoring systems, and other tutoring 
systems,” 214. 
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resolution) have also been explored.” One reason for the relatively low- 
levels of interest in ITS for ill-defined domains probably stems from the 
fact that imprecise problems often requite students to apply cognitively 
complex skills, while the contexts can be uncertain and dynamic, all of 
which can be challenge to model in traditional ITS. The relative lack of 
structure also makes it difficult to scaffold effective learning pathways 
without artificial constraints, to provide appropriate feedback, and to 
evaluate what learning is actually happening. ITS in ill-defined domains 
can also require additional pedagogical approaches, such as non-didactic 
Socratic dialogue, collaborative activities, or exploratory learning (which 
we look at in more detail later). 

As we have noted, because what AIED looks like and can do is still 
emerging, in this chapter we are going to introduce a wide range of 
AIED examples. Our list is far from definitive, but it will indicate the 
broad areas of AIED research, and it will highlight the many possibilities 
and challenges introduced by the application of AI technologies in 
classrooms. In this section, we are discussing ITS—so we begin with 
some prominent current ITS examples, most of which focus on 
structuted domains (such as mathematics): Carnegie Learning’s Mathia, 


; ; s 
Worcester Polytechnic Institute’s Assistments, and Knewton’s alta. 


Mathia 


Building on research at Carnegie Mellon University, Mathia” (previously 
known as Cognitive Tutor) delivers Al-driven personalized mathematics 
instruction for K-12 students. As the students work through carefully 
structured mathematics tasks, the system acts as a personal coach, 
monitoring their progress (their successes and misconceptions) and 


233 Lynch C., et al. (2006). “Defining “ill-defined domains”; a literature survey.” In (2006), Proceedings of the 
Workshop on Intelligent Tutoring Systems for Ill-Defined Domains at the 8th International Conference on 
Intelligent Tutoring Systems. http://people.cs.pitt.edu/~collinl/ Papers /Ill-DefinedProceedings.pdf; Woolf, 
B. (2010). “Social and caring tutors.” ITS 2010 keynote address. l 
https://link.springer.com/chapter/10.1007/978-3-642-13388-6_5; Lane, C., et al. (2007). Intelligent tutoring 
for interpersonal and intercultural skill.” http://ict.usc.edu/pubs/Intelligent Tutoring for Interpersonal and 
Intercultural Skills.pdf l l 

24 https://www.carnegielearning.com/products/software-platform/mathia-learning-software 
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directing them along individualized learning pathways. It also provides 
automatic feedback that aims to explain not just why the individual 
student got something wrong but also how they might get it right. 
Interestingly, Carnegie Learning argues that Mathia is most effective 
when it is used as part of a blended learning approach (i.e. they 
acknowledge that, on its own, it is insufficient), which includes the use of 
both print and digital resources, and involves students learning 


collaboratively in groups as well as individually. 
Assistments 


Our second example of a current instructional ITS is Assistments,”*° 
developed at Worcester Polytechnic Institute, which overall uses a similar 
approach to Mathia. However, Assistments also aims to address a key 
issue for ITS, that they by definition leads to students progressing at 
different rates, meaning that in any one classroom the students can be at 
increasingly diverging levels of attainment (potentially making the 
classroom teachet’s job more, rather than less, challenging). Accordingly, 
Assistments is designed to help students catch up in the evenings, 
working independently at home, so that in the classroom everyone’s 
progress remains roughly aligned. Both Mathia? and Assistments”” have 


strong, although not definitive,” evidence for their effectiveness. 


alta 


Our third example ITS is Knewton’s alta,” which is doubly unusual: 
it is designed for Higher Education students and it focuses on a range of 
subjects, including mathematics, economics, chemistry, and statistics. 
Nonetheless, like most ITS, alta aims to function like a 1:1 tutor, with 


personalized, step-by-step instruction, assessment, feedback, and just-in- 


235 https://www.assistments.org 

6 Pane, J.F., et al. (2015). “Continued progress. Promising evidence on personalized learning.” 
https://www.rand.org/content/dam/rand/pubs/research_reports/RR1300/RR1365/RAND_RR1 365.pdf 
37 Roschelle, J., et al. (2017). How Big Is That? Reporting the Effect Size and Cost of ASSIST ments in the Maine 
Homework Efficacy Study. SRI International. 

238 Holmes, W., et. al. Technology-enhanced Personalised Learning, 65 & 68. 
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time remediation while a student engages with an assignment. The alta 
approach clearly maps onto the typical ITS architecture outlined earlier. 
For each subject, it has a domain model, which uses open educational 
resources” (OER) and includes tutor-selectable learning objectives, 
together with a semantic network (or knowledge graph)” of 
relationships between the content and objectives. The domain models 
also include databases of relevant questions, together with data about the 
difficulty of those questions (based on how previous students have 
performed when responding to them), Alta’s pedagogy model is based on 


%2 (it works at the granularity of individual 


item response theory 
questions, taking into account both the question’s difficulty and their 
representativeness of the underlying concepts). It also adopts a mastery 
level approach (students do not move onto new learning objectives until 
they have achieved mastery of earlier learning objectives). In particular, 
the model assumes that if a student masters one of two learning 
objectives that are related according to the domain model’s knowledge 
graph, there is a good chance that they have also mastered the other one. 
Meanwhile, alta’s learner model represents a student’s level of mastery in 
terms of the learning objectives at any given point in time. This is based 
on the observed history of the individual student’s interactions, and of all 
student interactions, including which questions the students have 
answered correctly and incorrectly, giving more weight to an individual 


student’s most recent responses. 


Some Final Examples 


In addition to these three examples of current ITSs, there are many 
others. Meanwhile, as we have repeatedly commented, new ones seem to 
appear all the time, such that any list will always be incomplete. With this 


in mind, we will round out our discussion by briefly mentioning just four 


240 Hylén. J. (2006). “Open educational resources: Opportunities and challenges.” Proceedings of Open Education, 
49-63. 

241 Paulheim, H. (2016). “Knowledge graph refinement: A survey of approaches and evaluation methods.’ , 
Semantic Web 8 (3): 489-508. https://doi.org/10.3233/SW-160218 

242 Embretson, S.E. and Reise, S.P. (2013). Item Response Theory. Psychology Press. 
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mote, perhaps less well-known, but widely available ITS (chosen because 
they adopt slightly different approaches): Area9 Lyceum’s Rhapsode, 
Dreambox, Toppr, and Yixue. We could also have chosen ALEKS,” 
Bjyu,”* Century,” CogBooks,””* iReady,”” Realizelt,”* Smart Sparrow,” 
Summit Learning,” or ... the list goes on. 

Area9 Lyceum’ stands out. They develop an ofganization’s existing 
learning materials into adaptive content that is delivered on their 
platform. Like all ITS, their approach aims to match the learning content 
and pathway to the needs and skill level of each individual learner, but 
the platform also uses an approach they call “continuous self- 
assessment.” This involves learners rating how confident they are in their 
response to each question and task, which is then used to further adapt 
the learner’s experience (even if they give a correct answer to a question, 
if they are not confident in that answer, they will be given additional 
related learning support). The following diagram shows how adaptive 
learning simultaneously reduces the time spent on learning for the bulk 
of the students, while allowing the slower ones to achieve mastery at their 


own pace. 


CSE 


28 https://www.aleks.com 

244 https://byjus.com 

4 https://www.century.tech 

246 https://www.cogbooks.com 

247 https://www.curriculumassociates.com/Products/i-Ready 
28 http://realizeitlearning.com 

29 https://www.smartsparrow.com 

250 https://www.summitlearning.org 
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Reduction in time achieved by using adaptive learning 
instead of traditional digital resources. 


* A traditional e-learnin 
i g 


. course took4§ minutes io 
* complete 
* 


By using adaptive 
fearning, this was 
reduced by up to 60%. 


TIME IN HOURS 


Source: Area9 Lyceum, private communication. 


Our second example, Dreambox,” aims to provide students with 
personalized learning pathways, in K—8 mathematics: “the right next 
lesson, at the right level of difficulty, at the right time.” Again adopting a 
typical ITS approach, their Al-driven technology collects more than 
48,000 data points every hour a student engages with the system, which it 
uses to evaluate the strategies the student employed to solve the 
problems, to adjust the lesson’s level of difficulty, to sequence the lessons 
and to provide hints. Of the commercial ITS, Dreambox is fairly unusual 
in encouraging independent evaluations, with a recent study conducted 
by Harvard University finding that “the evidence for the causal impact of 
DreamBox on student achievement is encouraging but mixed.” 

Meanwhile, Toppt’* is an India-based company that offers 
personalized learning ITS mobile apps, across a wide range of school 
ages and subjects (from history to accounting). It uses machine learning 
to map out a student’s strengths and weaknesses, based on their previous 
behavior, in order to personalize questions and adjust the speed of 
presentation to make the experience optimal for each individual. This 


252 http://www.dreambox.com 

253 Fullerton, J. (2016). “Dreambox learning achievement growth in the Howard county public school system 
and rocketship education.” Center for Educational Policy Research. https://cepr.harvard.edu/dreambox- 
learning-achievement-growth 
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ITS prediction system is complemented by a novel AI-dtiven technology 
that is, they say, designed “to solve doubts.” Students can upload an 
image of a topic about which they are unsure, which a bot checks against 
a growing database of other uploaded doubts and solutions, 

Our final example ITS is Yixue,” which styles itself as the first 
intelligent adaptive education system in China. Again as a typical ITS, 
Yixue aims to simulate a teacher, providing students with a personalized 
learning plan and one-to-one tutoring. Drawing on standard textbooks, 
Yixue have broken each of its various subjects into around 10,000 
separate knowledge points, which ate used to benchmark an individual 
student’s understanding and capabilities, so that the system can predict 
which materials and pathway will be most effective. 


Dialogue-Based Tutoring Systems 


We began our discussion of ITS with Jaime Carbonell’s SCHOLAR.” 
However, SCHOLAR is in at least one sense unlike most of the ITS that 
we have so far explored. Rather than presenting an individualized 
sequence of instructional material or learning activities (as is typical of 
ITS), SCHOLAR engaged students in conversations about the topic to 
be learned. This aspect gave tise to a version of ITS known as dialogue- 
based tutoring systems (DBTS). However, as with ITS, what constitutes 
DBTS is fuzzy-edged. Instead, we will again introduce some prominent 
examples : CIRCSIM, AutoTutor, and Watson Tutor. 


CIRCSIM 


One of the earliest DBTSs was CIRCSIM,”” which was developed in the 
1980s at the Illinois Institute of Technology in partnership with Rush 
Medical College. It was designed for first-year medical students, to help 
them learn about the baroreceptor reflex control of blood pressure. 
CIRCSIM used one-to-one tutorial dialogues, involving some limited 


255 http://www.classba.cn 
56 Carbonell, J.R., “AI in CAI.” 190-202. 
237 Evens, M. and Michael, J. (2006). One-on-One Tutoring by Humans and Computers. Psychology Press. 
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natural language processing and natural language generation, on the 
assumption that real understanding of something involves being able to 
talk articulately about it. It also used a rule-based expert systems 


approach, implementing conditional rules such as: 


— Ifthe student answer is correct, then proceed. 
— Ifthe student answer is partially correct, then give 
acknowledgement and proceed. 
— If the student answer is a ‘near miss’, then 
introduce a nested method. 
— If the student answer is “don’t know”, then give 
answer and proceed.258 
Interestingly, CIRCSIM was not designed to introduce students to the 
topic. Instead, students were expected to have already acquired the facts 
and concepts ftom their readings and lectures. Their dialogue with the 
system helped the students explore in depth, better understand and 
consolidate what they had already learned. With this aim, students were 
asked to solve problems while engaging in an iterative typed dialogue. 
They would begin with a mandatory, guided virtual experiment. The 
program then directed the students step-by-step through a sequence of 
eight procedures, guiding them to predict outcomes based on supplied 
data, and to develop a simplified model for the homeostatic baroreceptor 
reflex system. Throughout, the emphasis was on the need for developing 
a chain of causal reasoning in solving this and similar problems. 


AutoTutor 


Our second example, AutoTutor,”’? which has been extensively 
researched for over twenty years, is probably the most influential DBTS. 
Developed at the University of Memphis, it simulates a tutorial dialogue 
between human tutors and students as they work step-by-step through 
online tasks (most often in computer science topics, but also in physics, 
biology, and critical thinking). The aim is to encourage students to 


258 Tbid, 45. 
* Graesser, A,C., et al. (2001). “Intelligent tutoring systems with conversational dialogue.” AI Magazine 22 (4): 
39. 
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develop detailed responses and an in-depth understanding, rather than 
the short responses and shallow knowledge that can be the outcome of 
some step-by-step instructional ITS. 

AutoTutor uses a statistical technique known as latent semantic 
analysis (LSA) to compare — student-written speech with a 
multidimensional matrix of concepts drawn from a large corpus of 


relevant textbooks,?” 


This matrix of concepts and a curriculum script 
(comprising example questions, problems, diagrams, declarative 
knowledge, and good and bad responses) effectively constitutes 
AutoTutor’s ITS domain model. 

Meanwhile, its version of a pedagogy model comprises Socratic 
tutoring principles (probing with questions rather than providing 
instruction) and classroom-based tutorial ptactices (based on analyses of 
dialogue from more than 100 hours of face-to-face human tutorials). Its 
adaptive learning activities involve engaging the student in a tutorial 
dialogue, a developing conversation in which the students are guided 
towards discovering for themselves a correct solution for the current 
problem. 

An AutoTutor tutorial dialogue typically comprises five steps: (1) 
AutoTutor (sometimes represented by an online animated character) 
poses a question or problem, (2) the student attempts to answer, typing 
their response into the system (or, with some versions, speaking aloud), 
(3) the tutor determines whether the student understands the target 
concept, by assessing how closely their contribution matches the concept 
as expressed in the textbook corpus (the LSA approach means that the 


oo 


%0 Graesser, A.C., et al. (2000). “Using latent semantic analysis to evaluate the contributions of students in 
AutoTutor.” Interactive Learning Environments 8 (2): 129-47, https://doi.org/10.1076/1049-4820(200008)8:2;1- 
B;FT129. Latent semantic analysis (LSA), developed by Thomas Landauer (University of Colorado) originally 
for indexing documents for information retrieval, is “both a computational model of human knowledge 
representation and a method for extracting semantic similarity of words and passages from text”. Peter W. 
Foltz, Darrell Laham, and Thomas K. Landauer. (1999). “The intelligent essay assessor: Applications to 
educational technology.” Interactive Multimedia Electronic Journal of Computer-Enhanced Learning 1 (2). 
http://www.imej.wfu.edu/atticles/1 999/2/04/printver.asp 

21 Graesser, A.C. (2011). “Learning, thinking, and emoting with discourse technologies.” American Psychologist 
66 (8): 746-57. https://doi.org/ 10.1037 /a0024974 
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student does not have to match specific wording but only the underlying 
concept), (4) if the tutor decides the student has not sufficiently 
understood the concept, the tutor provides feedback (in the form of 
corrective statements, questions, prompts, hints or summaries), perhaps 
to correct a misunderstanding or to encourage the student to extend their 
answer, and (5) steps two to four are repeated, with the aim of helping 
the student improve their answer. Finally, AutoTutor’s version of a 
student model comprises the student’s contributions to the dialogue, 
together with the goodness of fit (the closeness of the student’s answer 
to the corpus of texts) of each of those student inputs.” 
An AutoTutor dialogue”? might look something like this: 


AutoTutor <Main Question>: Suppose a boy is in a free- 
falling elevator and he holds his keys motionless in front 
of his him and he lets go. What will happen to the keys? 
Explain why. 

Student: The keys would not move but hover at the start 
point. 

AutoTutor <Pump for information>: I bet you can 
explain this a little bit more. 

Student: The velocity of the keys is the same due to 
gravity. 

AutoTutor <Hint>: What about the acceleration of the 
objects involved? 

Student: What is acceleration again? 

AutoTutor <Answer>: Let me try to answer your 
question. Acceleration is the rate that velocity changes 
over time, like getting faster or getting slower. Acceleration 
is a vector quantity, so it needs to have a direction. 
AutoTutor <Hint>: Let’s try this again. What about the 
acceleration of the objects involved? 

Student: The acceleration of the objects in the elevator are 
the same as the acceleration of the elevator which is the 
earth’s gravitational acceleration. 


26 Wiemer-Hastings, P., et al. (1998). “The foundations and architecture of AutoTutor.” Intelligent Tutoring 
Systems. Springer. 334-343, http://link.springer.com/chapter/10.1007/3-540-6871 6-5_ 39. 

263 Nye, B.D., Graesser, A.C., and Hu, X. (2014). “AutoTutor and family: A review of 17 years of natural 
language tutoring.” International Journal of Artificial Intelligence in Education 24 (4): 434 
https://doi.org/10.1007/s40593-014-0029-5 
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AutoTutor <Short Feedback>: Good job! 

AutoTutor <Prompt>: Try this. What direction are the 
objects going? The objects are falling <the animated 
character gives a downwards hand gesture to cue the 
student to express something>? 

Student: Downward. 

AutoTutor <Short Feedback>: Right. 

AutoTutor <Prompt-completion>: The objects are falling 
downward. 


Over its two decades, AutoTutor has been evaluated in multiple 
studies." In summary, these have shown that it can achieve average 
learning gain-effect sizes of approximately 0.8, with higher gains for deep 
learning (understanding concepts) than for shallow learning (knowing 
facts), suggesting that it is more effective than many non-expert human 
tutors and might in some situations be as effective (at least in terms of 


learning gains) as a typical expert tutor.” 
Watson Tutor 


A more recent DBTS is the Watson Tutor% being developed 
collaboratively by the global corporations IBM and Pearson, which at the 
time of writing is being evaluated before being released as a commercial 
product (integrated into a Pearson higher education courseware product 
called REVEL). Watson Tutor is a dialogue-based tutorial system using 
natural language conversation that guides students through a review 
session—in other words, like AutoTutor and CIRCSIM, it does not set 
out to introduce new knowledge but to enable a deeper understanding of 
existing knowledge.” As the students engage with Watson Tutor, it 


provides supportive content (such as text, images, and videos), tracks 


264 D’Mello, S. and Graesser, A. (2012). “AutoTutor and sffective AutoTutor: Learning by talking with 
cognitively and emotionally intelligent computers that talk back.” ACM Transactions on Interactive Intelligent 
Systems (TiS) 2 (4): 23. 

265 VanLehn, “The relative effectiveness of human tutoring, intelligent tutoring systems, and other tutoring 
systems”; Nye, Graesser, and Hu, “AutoTutor and family.” 

266 https://www.ibm.com/watson/education 

27 Ventura, M., Chang, M., Foltz, P., Mukhi, N., Yarbro, J., Salverda, A. P., ... Afzal, S. (2018). “Preliminary 
evaluations of a dialogue-based digital tutor.” In Carolyn, R. (ed.). Proceedings of the 19th International Conference. 
AIED 2018 London, UK, 480-483. 
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their progress, and adapts the conversation based upon a classification of 
the student’s responses and an assessment of their subject mastery. 

The Watson Tutor draws heavily on the approach developed by the 
AutoTutor researchers, although its domain model, the formalization of 
the knowledge and skills to be learned, is detived from a single textbook. 
It comprises a set of learning objectives and enabling objectives (sub- 
learning objectives that support a main learning objective), a knowledge 
graph of the relationships between the learning objectives, main 
questions and main answers, assertions (knowledge components of the 
main answers), and hint questions to elicit assertions (typically 600 main 
and hint questions are derived from the single textbook) and fill in the 
blank questions (based on the student assertions). All of this is derived 
from the chosen textbook by means of an automated analysis undertaken 
by the IBM Watson toolset (the AI as a service mentioned earlier, which 
includes natural language understanding and classification tools). This 
approach to the domain model means that it can, at least in theory, 
straightforwardly be adapted to any academic domain and its textbook 
(its initial implementation covered developmental psychology, 
physiological psychology, abnormal psychology, sociology, 
communication, and government). 

The pedagogy model again adopts a Socratic approach, implemented 
as a dialogue manager that controls when and which responses are 
delivered to a student. Starting from a learning objective, Watson Tutor 
begins by asking the student a main question, with the aim of eliciting 
from the student the main answer. The student’s responses are classified 
by the IBM Watson system, enabling the system to respond with 
appropriate feedback or a hint question to help the student work towards 
an ideal answer. For main questions, the classifier matches the student’s 
response to all the knowledge components of the main answer, using a 
supervised machine learning approach that requires only 35 scored 
student answers per question. If the student struggles on a main learning 
objective, the Tutor will move to an enabling objective, repeating the 
dialogue approach. 
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The Watson Tutor’s learner model includes an estimation of each 
student’s proficiency over the course of the conversation, in terms of 
both the learning and enabling objectives. This also functions as an open 
learner model, since the student’s estimated proficiencies and developing 
performance can be inspected at a high level and with detailed 
granularity, by the individual students and their teachers. 

A Watson Tutor dialogue might look something like this: 


Watson Tutor <Learning objective main question>: Let’s 
review this topic... Try answering this question: What are 
some variations in the definition of family? 

Student: A family can be defined as people who live 
together. 

Watson Tutor <Feedback on student assertion>: 
Excellent. Kinship systems consist of the social links and 
boundaries, defined by biology and social custom, that 
establish who is related to whom. Let’s take it a bit further. 
Watson Tutor <Learning objective hint question>: How 
do social scientists define the family? 

Student: Social scientists define the family as a group of 
related people that reside together. 

Watson Tutor <Feedback on student assertion>: That 
looks partly correct. There’s a little more to the answer. 
Let’s work this out together. 

Watson Tutor <Fill in the blank hint>: Can you fill in the 
missing words? Social scientists typically define family as a 
group of people who live together in a household and 
share and/or ties. 


Exploratory Learning Environments 


An alternative to step-by-step instruction by ITS and DBTS step-by-step 
dialogue is provided by a third type of AIED, known as exploratory 
learning environments (ELEs). ELEs adopt a constructivist approach. 
Rather than following a dynamically generated step-by-step sequence, 
students ate encouraged to actively construct their own knowledge by 


exploring and manipulating elements of the learning environment. 


268 Based closely on Ventura et al., 482. 
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In fact, exploratory and discovery learning have been around for a 
long time,” but they remain controversial.” Critics argue that, because 
there is no explicit instruction and students are expected to discover 
domain principles for themselves, it causes cognitive overload and leads 
to poor learning outcomes. This is where AI comes in, with many recent 
ELEs including Al-driven automatic guidance and feedback, addressing 
misconceptions and proposing alternative approaches, to support the 
student while they explore. 

As we have seen, delivering effective Al-driven support requires a 
learner model. However, building learner models for unstructured 
environments like ELEs can be challenging: “The unconstrained nature 
of the interaction and the lack of easily definable correct behaviors make 
it difficult to know a priori what behaviors are conducive for learning.””” 
Nonetheless, student models are usually an important component of 
ELEs. 

Again, we will briefly explore four examples,” each of which includes 
a student model and use different Al-driven approaches to provide the 
necessaty support: Fractions Lab (which delivers automated feedback 
according to a student’s affective state), Betty’s Brain (which involves a 
teachable agent), Crystal Island (which uses a games-based approach), 
and ECHOES (which is designed to support children on the autism 
spectrum). 


2 E.g., Bruner, J.S. (1961). “The Act of Discovery.” Harvard Educational Review 31: 21-32. 

270 Kirschner, P., Sweller, J., and Clark, R.E (2006). “Why minimal guidance during instruction does not work: 
An analysis of the failure of constructivist, discovery, problem-based, experiential, and inquiry-based 
teaching.” Educational Psychologist 41 (2): 75-86. 

271 Fratamico, L., et al. (2017). “Applying a framework for student modeling in exploratory learning 
environments: Comparing data representation granularity to handle environment complexity.” International 
Journal of Artificial Intelligence in Education 27 (2): 321. https://doi.org/10.1007/s40593-016-0131-y 

272 Building on du Boulay, et al. (2018). “What does the research say about how artificial intelligence and big 
data can close the achievement gap?” In Enhancing Learning and Teaching with Technology, Luckin, R. (ed.). 
Institute of Education Press, 316-27. 
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Fractions Lab 


EE 


Fractions Lab, which was developed by an EU-funded research 


project,” 


is designed to help students develop conceptual knowledge, 
the underlying principles, of fractions. In this ELE, students can choose 
and manipulate fraction representations (for example, having chosen a 
rectangle, jug or number line to represent a particular fraction, they can 
create the fraction by changing the numerator and denominator), the aim 
being to solve a given fractions problem (such as “Use the Fractions Lab 


tools to add together 7 and =") To avoid cognitive overload while they 


address the given task, Fractions Lab uses Al-techniques to provide the 
students with adaptive support—that is to say, feedback or guidance 
specific to where they are in their attempted solution (such as “To add 
the two fractions together, you first need to make them equivalent. How 
do you need to adjust the denominator?”), However, in addition to 


providing this context-specific guidance, the feedback also aims to 


273 http://www. italk2learn.eu 
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enhance the student’s affective states that is, to move students from 
nominally negative affective states (such as frustration or boredom) into 
nominally positive affective states (which are usually assumed to be more 
conducive for learning). 

This is achieved by means of Bayesian networks trained with data 
from classroom studies, one of which determines the most appropriate 
type of formative feedback to be given to the student. For example, if the 
student is confused, the Bayesian network determines that an affect 
boost (such as “Well done. You're working really hard!) or specific 
instructive feedback (such as “Use the comparison box to compare your 
fractions.’’) is most effective. 

Other feedback provided by the system includes Socratic feedback 
(such as “What do you need to do now, to complete the fraction?”), 
reflective prompts (such as “What do you notice about the two 
fractions?”), affirmation prompts (such as “The way that you worked that 
out was excellent”), and sequence prompts (such as “Are you sure that 
you have answered the task fully? Please read the task again”). 

Fractions Lab’s pedagogy and domain models comprise both the 
overall constructivist ELE approach, and the information used to 
determine the content of the formative feedback. Meanwhile, the learner 
model includes data about the student’s inferred affective state, their 
progress with the current task, their interactions with the learning 
environment (whether a representation has been created, selected or 
manipulated), the type of feedback they have received, and the specific 
message, and whether or not the student follows the feedback. While the 
student interacts with the fractions representations to answer the tasks, 
the learner model is constantly being updated with information, both 
about those interactions and with the feedback that has been provided to 
the student. 
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As part of a larger project,’”* Fractions Lab was evaluated in schools in 
Germany and the UK, comparing the effectiveness of Fractions Lab in 
combination with an ITS. Although the tools were only used for a short 
time, the outcomes showed that the combination of ELE and ITS 
achieved a learning gains effect size of 0.7 (compared with the ITS 
alone), suggesting that Al-supported ELEs can offer a useful approach to 


learning. 
Betty’s Brain 


An iconic ELE is Betty’s Brain,” which involves an Al-driven, teachable 
agent. It was designed to facilitate the learning of scientific conceptual 
understanding, using river ecosystems as a use-case exemplar. What 
distinguishes Betty’s Brain is that, through their engagement with the 
system, students are encouraged to teach a fellow student, in fact a virtual 
agent, called Betty. This approach (which is the foundation of the 
system’s pedagogy model) has been adopted because learning by teaching 
has been shown to be effective—it is known to help students structure, 
reflect on and develop a more in-depth understanding of whatever is 
being learned.””° 

Within an overarching narrative (helping Betty to join a science club), 
students are supported to teach Betty, then to query Betty to see how 
much she has understood, and finally to quiz Betty to see how well she 
does on questions generated automatically by the system, many of which 
the student may not have considered. 

The mechanism used for teaching Betty centers on a concept map 
editor, which represents what the student has taught Betty. Drawing on a 
range of provided reading materials and using the available editing tools, 


the student builds a conceptual map of the river ecosystem (the 


4 Rummel, N., et al. (2016). “Combining exploratory learning with structured practice to foster conceptual 
and procedural fractions knowledge.” In Looi, C.K., Polman, J., Cress, U., and Reimann, P. (eds.) Transforming 
Learning, Empowering Learners: The International Conference of the Learning Sciences 1: 58-65. 
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relationships between a rivers plants, animals, microorganisms, chemical 
components, and physical characteristics), attaching nodes (representing 
particular knowledge components) via edges (representing causal and 
other links between the various components). In effect, the students 
build their own semantic network, which forms the core of the system’s 
learner model (i.e. it represents the students current knowledge and 
understanding). The learner model also includes a record of the student’s 
interactions with the system. Interestingly, the concept map, because it is 
visual and open to inspection by the student and the teacher, also 
functions as an open learner model. 

Once Betty has been taught, the student can ask her a question (such 
as, “If macroinvertebrates increase what happens to bacteria”). In 
response, Betty reasons using the concept map to generate an answer 
(such as, “An increase in macroinvertebrates causes no change in 
bacteria”). The student can also ask Betty for an explanation, which Betty 
gives while highlighting the causal paths in the concept map. 

The system also uses the concept map, together with the system’s 
domain model, to generate the quiz questions that are administered by 
Mr. Davis, a virtual teacher. Betty’s responses to the quiz questions draw 
directly from the concept map, while Mr. Davis’s feedback also uses the 
domain model to suggest how the student might edit the concept map in 
order to help Betty achieve a higher quiz score—in other words, to 
correct any mistakes made by Betty and to identify any misconceptions. 
Mr. Davis also makes suggestions at the meta-cognitive level, for example 
about making better use of the reading materials, in an effort to help the 
learner develop good meta-learning strategies (study skills). 


Betty’s Brain has been evaluated in multiple studies,” 


although in its 
original configuration student outcomes were often split, almost half and 


half, between those who made very good progress with the system and 


27 Biswas, G., Segedy, J.R., and Bunchongchit, K. (2016). “From design to implementation to practice a 
learning by teaching system: Betty’s Brain.” International Journal of Artificial Intelligence in Education 26 (1): 350- 
364. 
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those who struggled. The researchers went on to develop a new version, 


which they used to investigate different learning behavior profiles.” 


Crystal Island 


Crystal Island*” emerged from research at North Carolina State 
University. It takes an immersive, first-person, computer-game approach, 
with students playing the role of a detective investigating a mysterious 
disease on a remote island. This games-based learning approach? 
functions as Crystal Island’s pedagogy model. In solving the mystery, 
students use and develop their literacy skills while gaining experience of 
professional scientific inquiry methods (including evidence gathering, 
hypotheses testing, and data analysis), all of which together constitute the 
domain model. Meanwhile, the students’ developing knowledge, their 
affective states and their skills, are automatically modeled (in the ELE’s 
learner model) and they receive automated supportive feedback. In 
addition, throughout the gameplay, they engage with Al-driven 
autonomous non-player characters (companion agents), which build on 
AI techniques developed over many years in mainstream computer 


games," 


ECHOES 


Our fourth example ELE, ECHOES,” again involved a games-based 
approach, but this time to support young children who ate on the autism 
spectrum. ECHOES was a virtual environment, a magic garden, in which 
the child interacted with an intelligent child agent called Andy. The 
child’s teacher (not AT) selected one of twelve learning activities, led by 


78 Jeong, H., et al. (2008). “Using hidden Markov models to characterize student behaviors in learning-by- 
teaching environments.” In Intelligent Tutoring Systems, 614-25, https://doi.org/10.1007/978-3-540-69132-7_64 
*” http://projects.intellimedia.ncsu.edu/ctystalisland 

* Holmes, W. (2017). “Digital games-based learning. Time to adoption: Two to three years?” In Education and 
New Technologies: Perils and Promises for Learners. Sheehy, K. and Holliman, A.J. (eds.). Routledge. 

281 'Yannakakis, G.N. and Togelius, J. (2018). Artificial Intelligence and Games, Springer International Publishing. 
https://doi.org/10.1007/978-3-319-63519-4 

28 Bernardini, S., Porayska-Pomsta, K., and Smith, T.J. (2014). “ECHOES: An intelligent serious game for 
fostering social communication in children with autism.” Information Sciences 264 (April): 41-60. 
https://doi.org/10.1016/j.ins.2013.10.027 
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Andy, each of which was designed to enhance the child’s capacities for 
joint attention and to help them develop their social communication 
skills. 

The magic garden was displayed on a large touchscreen monitor, 
allowing the child and Andy to interact with each other and with objects 
in the garden. Sometimes, when touched, the garden objects transformed 
in unusual ways. For example, tapping the petals of a flower could turn it 
into a floating bubble or a bouncy ball. The system also included an eye- 
tracking camera, allowing Andy to “know” where the child was looking. 

Andy was designed to function as an artificially intelligent social 
partner who could act both as a peer and as a tutor. Its implementation 
was based on a well-established AI agent architecture called FAtiMA,2® 
which enabled it to be autonomous, proactive, reactive, and socio- 
emotionally competent. In particular, Andy was designed to be a positive 
and supportive character. For example, he always greeted the child by 
mame, gave positive feedback when the child participated in an 
interaction, and tried to re-engage the child if they appeared distracted. 
Andy also used facial expressions and gestures to indicate his emotional 
responses. For example, he smiled and gave a thumbs-up when the child 
initiated an activity. 

ECHOES also included a pedagogy model, which monitored the 
developing interactions between the child and Andy, to help ensure that 
the learning objectives were achieved, and a user model that aimed to 
monitor the cognitive and emotional state of the child, so that Andy 
could give appropriate real-time feedback. 


Summary 


As we have seen, because ELEs are unstructured and open-ended 
learning environments that students can explore as they like, there is no 
clear definition of correct behaviors, which makes it difficult to model 
the student and to provide the necessary guidance. With this in mind, 


*83 Dias, J., and Paiva, A. (2005). “Feeling and reasoning: A Computational model for emotional characters.” 
In Progress in Artificial Intelligence, 127-40. Springer. https:// doi.org/10.1007/11595014_13 
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over several years, Conati and colleagues have developed and researched 
a multilayered, student-modeling framework,’ which they implemented 
in an ELE called CCK, which can potentially be applied in other ELEs. 
This framework uses multiple, logged student actions to learn which 
behaviors should trigger remedial guidance, and which lead to what 
outcomes (such as high- or low-achieving). It involves the use of 
unsupervised-learning to cluster groups of students who, based on their 
logged data and learning outcomes, learn similarly. The logged data 
includes the components used (such as a light bulb), student actions 
(join), and simulation outcomes (a change in light intensity). The student- 
clusters model is then used to classify new students, and to trigger real- 
time, adaptive support based on the logged and thus anticipated 
behaviors, in order to support students to achieve higher learning 


outcomes. 


Automatic Writing Evaluation 


The AIED applications we have looked at so far—the step-by-step 
instructional and dialogue-based systems, and the exploratory learning 
environments—all involve students working on computers (sometimes 
mobile devices), following individualized learning pathways while 
receiving immediate adaptive support. Another type of AIED, automatic 
writing evaluation (AWE), uses natural language and semantic processing 
to provide automatic feedback on student writing submitted to the 
system.**° 


28t Kardan, S. and Conati, C. (2015). “Providing adaptive support in an interactive simulation for learning: an 
experimental evaluation.” In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, 
3671-3680. https://doi.org/10.1145/2702123.2702424 

*85 Pujitani, S. and Minemura. K. (2017). “An analysis of expectations for artificial intelligence-supporting 
software in mobile learning.” 
https://www.researchgate.net/publication/324537420_An_Analysis_of_Expectations_for_Artificial_Intellige 
nce-supporting_Software_in_Mobile_Learninght 

28 According to John Behrens (Pearson), automated essay grading is one area that “machine learning is 
starting to make progress.” Quoted in Johnson, S. (2018). What Can Machine Learning Really Predict in Education? 


https://www.edsurge.com/news/201 8-09-26-what-can-machine-learning-really-predict-in-education 
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Broadly speaking, there are two overlapping AWE approaches, 


formative (providing support to enable a student to improve their writing 


287 


before submitting it for assessment)” and summative (automatic 


scoring). However, in line with the long established automated grading 
of multiple choice and fill-in-the-blank tests, much of the work is 
directed to scoring over feedback. It has often been driven by the desire 
to improve the cost, reliability, and generalizability of summative 
assessments, whether for use in small-scale settings by teachers (for low- 
stakes classroom assessment) or large-scale settings by testing companies 


(for large-scale, statewide, or national, high-stakes assessments). 


Efficiency is where the automated readers excel. The e- 
rater engine can grade 16,000 essays in about 20 seconds, 
according to ETS. An average teacher might spend an 
entire weekend grading 150 essays, and that efficiency is 
what drives more education companies to create 
automated systems.?89 


This is why automated essay feedback and scoring is probably the 
best-funded area of AIED research, and why it has spawned a large 
number of commercial systems.”” There are so many available AWE 
systems,” with different approaches and limitations, that we will again 


simply introduce some prominent examples. 


PEG 


The beginnings of AWE can be traced to the development in 1966 of 
Project Essay Grade (also known as PEG) by Ellis Page at Duke 


287 One example being M-Write: https://Isa.umich.edu/sweetland/m-write.html 

288 One example being Gradescope: https://www.gradescope.com 

28 https://www.npr.org/sections/alltechconsidered/2012/04/24/151308789/for-automatic-essay-graders- 
efficiency-trumps-accuracy?t= 1542533112695 

2% Dikli, S. (2006). “An overview of automated scoring of essays.” The Journal of Technology, Learning and 
Assessment 5 (1). https://ejournals.bc.edu/ojs/index.php/jtla/article/view/ 1640. Raczynski, K., and Cohen, A. 
(2018). “Appraising the scoring performance of automated essay scoring systems—some additional 
considerations: Which essays? Which human raters? Which scores?” Applied Measurement In Education 31 (3): 
233-40. https://doi.org/10.1080/08957347.2018.1464449. See also, Hubert’s “AI in education—automatic 
essay scoring.” https://medium.com/hubert-ai/ai-in-education-automatic-essay-scoring-6eb38bb2e70 

31 Stevenson, M. and Phakiti, A. (2014). “The effects of computer-generated feedback on the quality of 
writing.” Assessing Writing 19: 51-65. 
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University. The original version of PEG used correlational statistics to 
compare submitted essays with a training set of up to 400 comparable 
essays that had already been marked by teachers, an approach that was 
shown in various studies to achieve predicted scores comparable to the 
human markers. However, the PEG system was criticized for focusing 
on indirect measures of writing skill (the surface features of essays such 
as the number of sentences, use of punctuation and grammar) rather than 
on the meaning of the sentences, the writing style, or how the writer had 
developed their arguments (in other words, it was criticized for focusing 
on the form rather than the content of the essays). For this reason, PEG 
was incapable of providing meaningful formative feedback, to enable 
students to improve the academic (rather than the surface) quality of 
their essay, and instead provided only a summative mark. The 
effectiveness of the system was also dependent on the selection of the 
training essays, and the quality of the assessments made by the training 
set’s human markers. More recently, PEG was re-engineered to include 
computational linguistics, machine learning, and natural language 
processing techniques. It was also included in the Hewlett Foundation- 


sponsored Automated Student Assessment Prize competition.” 


Intelligent Essay Assessor 


An alternative early AWE approach, Intelligent Essay Assessor (IEA), 
used latent semantic analysis (LSA), the statistical technique that we 
introduced when discussing the dialogue-based tutoring system 
AutoTutor. LSA enables IEA to infer the meanings of words and 
sentences, by considering the context in which they occur, and to 
calculate the semantic relatedness of a target document to the training 
corpus: “The underlying idea is that the meaning of a passage is vety 


much dependent on its words and changing even only one word can 


22 Shermis, M. D. (2014). “State-of-the-art automated essay scoring: competition, results, and future directions 
from a United States demonstration.” Assessing Writing 20 (April): 53-76. 
https://doi.org/10.1016/j.asw.2013.04.001 
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result in meaning differences in the passage. On the other hand, two 
passages with different words might have a very similar meaning.” 

The IEA calculated similarity scores for essays when compared with a 
set of training texts, which included a large number of pre-scored student 
essays, expett model essays, and knowledge source materials (such as 
textbooks and academic papers). The system determined the essay’s mark 
by averaging the similarity scores. However, the comparison with the key 
domain-tepresentative texts also allowed the system to provide diagnostic 
and evaluative formative feedback across six areas: ideas and content, 
organization, sentence fluency, word choice, conventions, and voice. IEA 
was also able to detect some plagiarism (i.e., passages replicating text in 
the knowledge source materials) and collusion (similar passages appearing 
in more than one essay in a cohort), both of which (if only because of the 
scale involved) can be difficult for human markers to identify. IEA was 
also included in the Automated Student Assessment Prize competition.” 


WriteToLearn 
Over recent years, the IEA approach has been further developed by the 


international education company Pearson and incorporated in their 
product WriteToLearn.”° The current system draws on a broad range of 
AI techniques in order to provide both in-depth formative feedback and 
summative scoring. Essays are assessed against one ot more rubrics, 
using a supervised machine-learning approach involving a training set of 
around 300 representative essays that have been scored by humans. ‘The 
rubrics involve traits such as focus, development of ideas, organization, 
language and style, voice, sentence correctness, and sentence fluency. The 
system is also able to detect a variety of errors in the submitted essay, and 
thus to provide a series of specific prompts in terms of narrative, 
exposition, description, and persuasion, all scaffolded on the student’s 


writing performance and designed to enable the student to improve their 


293 Dikli, “An overview of automated scoring of essays,” 5. 
24 Shermis, M.D. (2014). “State-of-the-art automated essay scoring: Competition, results, and future directions 
from a United States demonstration.” Assessing Writing 20: 53-76. https:/ /doi.org/10.1016/j.asw.2013,04.001 


25 https://www.writetolearn.net 
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next draft. In addition, the system also gives a score to the writing, by 
assessing it against a rubric designed to represent characteristics of high- 
quality writing: ideas, organization, conventions, sentence fluency, word 
choice, and voice. A secondary component in the software can be used to 
assess and provide feedback based on students’ summarizations of given 
texts, which has been shown to help students build their reading 
comprehension skills. 

WriteToLearn has been evaluated in a number of studies. In one state- 
wide study,” involving more than 20,000 students and 70,000 
assignments, the students submitted an average of around four revised 
drafts (more than is typical in a traditional classroom setting) and 
improved their overall scores by almost one point out of a maximum of 6 
(effect sizes are not given). The improvements were shown to be in both 
basic writing skills and higher-level traits like ideas and voice. 


e-Rater 


A third AWE approach, originally known as e-Rater, which was 
developed by the Educational Testing Service, is widely used (for 
example, in GMAT tests and, in a more recent version, for Common 
Core Standards).”” Like the earlier systems, e-Rater analyses a large set of 
linguistic features (syntactic variety, topic content, and lexical and 
syntactic cues) that it automatically extracts from essays using natural 
language processing techniques. Algorithms then assign values for every 
feature identified in an essay, which are computed using linear regression 
and compared to a training set of essays scored by human experts, to 
predict a final score. ETS claims the psychometric validity of e-Rater 
scores across a range of subject areas, while accounting for cultural and 


second-language differences. 


295 Foltz. P.W. and Rosenstein, M. (2013). “Tracking student learning in a state-wide implementation of 
automated writing scoring.” In NIPS Workshop on Data Driven Education. 

27 Kukich, K. (2000). “Beyond automated essay scoring.” IEEE Intelligent Systems. 
https://doi.org/10.1109/5254.889104 
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Revision Assistant 


Three, short, final AWE examples are Revision Assistant, OpenEssayist, 
and AI grading. Perhaps best known for their antiplagiarism software, 
which automatically checks student writing against billions of internet 
documents and journal papers, the Turnitin corporation also now offers 
Revision Assistant. This system is designed to evaluate and provide 
formative feedback on short student essays (between 200 and 700 words) 
in a range of genres, which it achieves by using both supervised machine 
learning (involving training essays that have been scored by at least two 
human teachers) and unsupervised machine learning (involving a 
collection of thousands of unscored essays collected from students who 
have already used the system in classrooms). 

The Turnitin analysis works by representing the submitted essays in 
terms of large numbers of low-level textual features (such as word n- 
grams and essay length), and it uses multiple methods to compute a 
prediction. For example, one core analytical technique is to delete a 
sentence from the submitted essay to determine how that edit affects the 
predicted score. If the predicted score increases, the system infers that 
the sentence is strong for the particular trait being evaluated. This allows 
the system to automatically provide sentence-level formative feedback 
specific to the rubric in question, drawn from a pool of more than a 
thousand feedback comments authored by content experts, for each draft 
that a student submits. 

Classroom observations (at the time of writing there have been no 
published efficacy studies) have suggested that the system generates 
automated feedback that is both well received by students and aligned 
with improving scores. Turnitin argues that their automated essay 
feedback system not only provides more frequent formative feedback to 
students but also “allows the teacher to step back from the sometimes- 
adversarial red pen and engage with their class as guides and readers, 
modeling the interpretation of feedback alongside their students.” 
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OpenEssayist 

OpenEssayist,”* which was developed by The Open University and 
Oxford University in the UK, takes a somewhat alternative approach, 
again using natural language processing but focusing more on how the 
feedback is presented to the students so that it is easily actionable. The 
system’s linguistic analysis engine uses unsupervised algorithms to cluster 
key words, phrases, and sentences from the student’s essay. It then 
generates several external representations. For example, key words and 
phrases are presented in a word cloud, which can be explored and 
organized into groups. It also uses automatically generated animations 
and interactive exercises to encourage the student to reflect on the 
content of their essay. This aims to help the student improve their 
writing, while also promoting higher-order learning processes: self- 


regulated learning, self-knowledge, and metacognition. 


AI Grading 


Our final, un-named, example was developed to address the problem of 
marking essays for thousands of students on the EdX MOOC (massive 
open online courses) platform.” This system again uses an innovative 
machine-learning algorithm, which was trained with hundreds of teacher- 
graded essays, and configured with teacher-written rubrics.” However, 
the EdX system warrants only a brief mention because it does not appear 
to be currently available, and details are difficult to find. However, it is 
included here for two reasons: because it is likely that the further 
developments in MOOC approaches to teaching and learning will require 
some way to assess student contributions at scale, and because it helped 
catalyze a critical reaction to the whole project of automatic essay 


scoring. Criticisms have been neatly summarized, and comprehensively 


298 Whitelock, D., et al. (2013). “OpenEssayist: An automated feedback system that supports university 
students as they write summative essays.” http://oro.open.ac.uk/41844/ 

299 https://www.edx.org 

300 Reilly, E.D., et al. (2014). “Evaluating the validity and applicability of automated essay scoring in two 
massive open online courses.” The International Review of Research in Open and Distributed Learning 15 (5). 


http://www. irrodl.org/index.php/irrodl/article/view/1857 
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referenced, on the Professionals Against Machine Scoring Of Student 
Essays In High-Stakes Assessment website: 


Studies show that by its nature computerized essay rating 
is: trivial (rating essays only on surface features such as 
word size, topic vocabulary, and essay length), reductive 
(handling extended prose written only at a grade-school 
level), inaccurate (missing much error in student writing 
and finding much error where it does not exist), 
undiagnostic (correlating hardly at all with subsequent 
writing performance), unfair (discriminating against 
minority groups and second-language writers), and 
secretive (testing companies block independent research 
into their products).3" 


Finally, we should pose the diametrically reversed question. What 
happens when students have access to AI technologies that are capable 
of automatically writing (generating) high-quality essays (leading 
inevitably to an arms race for supremacy between automatic writing and 


302 


automatic assessment?) “ For now a moot point, but almost certainly not 


for long. 


301 _http://humanreaders.org/petition/index.php 

302 Much like the ongoing arms race between Al-generated fake news (such as 

https:// www.technologyreview.com/s/61 0635/fake-news-20-personalized-optimized-and-even-harder-to- 
stop) and AI tools to identify fake news (such as http://adverifai.com). 
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What Other AIED is Out There? 


As we saw in our discussion of ITS, any survey of applications of AI in 
education will always be incomplete, because new AIED applications 
using new ATED techniques are being launched every day. This follows 
the upsurge in general interest in AI and the many recent advances made 
possible by faster computer processors, large amounts of educational big 
data, and new computational approaches. In fact, because education has 
become a key focus for many AI developers (as we noted at the outset, 
the market for AIED is predicted to be worth $6 billion by 2024), a quick 
Google search will identify hundreds of AI products claiming to support 
students and improve learning outcomes. The EdTech consultancy 
GettingSmart recently published just such a search with a long list of 
thirty-two types of commercial “applications that are (or soon will be) 
making good use of machine learning to support better education.” 

In fact, as summarized in the following table, most existing ATED 
applications may be categorized in terms of five complementary 
dimensions: (i) the type of learners for which the AIED is designed, (ii) 
the learning domain that it covers, (iii) the learning approaches it 
facilitates, (iv) the learning support that it provides, and (v) the teaching 
support that it provides. AI might also be found at an institutional level 
(outside of learning), both in learning management systems (such as 
MOOCs) and school-management platforms (to deal with class 
timetabling, staff scheduling, facilities management, finances, 
cybersecurity, safety and security, and e-authentication). However, these 
administrative uses of AI in education are beyond the scope of this book. 
One key distinction to note between AIED technologies that are 
designed to support students directly (student-facing tools, such as the 
ITS, DBTS, ELE, and Automatic Writing Evaluation systems that we 
have discussed) and AIED technologies that are designed to support 
teachers to support students (teacher-facing tools). We will return to this 
distinction later. 


303 http://www.gettingsmart.com/201 8/08/32-ways-ai-is-improving-education 
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Dimension 


Examples _ | 
| Type of learners Early years. | 
K-12. 
Higher education. 
Informal. i 
Professional. | 
| Students who have additional needs. 
_ Learning From maths and physics, to language learning | 
_ domain and music, and beyond. | 
Learning Step-by-step instructional adaptive learning. | 
approach Dialogue-based adaptive learning. | 
| Exploratory learning. | 
Writing analysis. | 
| Learning Learning diagnostics. | 
support Mentoring. | 
Assessment. | 
Network connectors. | 
Chatbots. | 
` Teaching Automatic learner profiles. | 
` support Smart gradebooks. | 


Here, however, we will conclude our look at applications of AI in 
education with six further sets of tools or technologies, some of which 
build on the AIED approaches that we have already mentioned, while 
others adopt alternative AI techniques. We begin with what we can only 
think to call ITS+, then Al-supported language learning, chatbots, 


augmented and virtual reality, and learning network orchestrators. 
ITS+: ALT School, ALP, and Lumilo 


By ITS+ we mean approaches that augment or extend standard ITS 
functionalities, that perhaps expand the reach of an ITS or add another 
layer. Our first example is is ALT School, a Silicon Valley venture 
founded by a former Google executive and funded by the Chan 
Zuckerberg Initiative. What sets ALT School apart from conventional 
schools is theit use of a big data ITS approach to deliver individualized 


3% http://www.altschool.com 
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learning to students throughout the whole school—in other wotds, this is 
in effect a school-wide ITS. Each week, all ALT School students are 
given an automatically generated individual playlist of activities that are 
designed to develop student mastery, and while they engage with the 
activities a vast range of data about their interactions is recorded and 
analyzed. The results, which include each student’s strengths, weaknesses, 
and progress, are made available to the teachers. Meanwhile, video 
footage of student activities, captured by classroom wall-mounted 
cameras, is also analyzed using AI techniques, to provide indicators of 
student engagement. Interestingly, ALT School appears recently to have 
pivoted their business model. Perhaps inspired by our next example, they 
now offer their technologies to other schools rather than focusing on 
running theit own. 

Our second ITS+ is ALP (adaptive learning platform), by 
KĶKidaptive,” which offers an “AIED engine as a service” to the 
developers of educational technologies that are not themselves enabled 
with Al—in other words, ALP offers backend ITS functionality for 
standard edTech. Partner companies connect their EdTech products to 
ALP, using client- or server-side software development kits (SDKs), 
which then analyzes their user data in real time. Data streams from a 
variety of learning contexts can be aggregated to create in-depth 
psychometric profiles (learner models) of each individual student’s 
interactions, preferences, and achievements. It then uses an item 
response theory, psychometric framework to determine the student’s 
optimal next challenge, instructional material, or activity, which is then 
delivered to the student by the partner’s EdTech product. ALP also 
provides personalized insights and recommendations to teachers and 
parents about the best ways in which they can support individual 
learners. 


35 http://kidaptive.com 
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We have left possibly the most intriguing ITS+ for last: Lumilo.*°° So 
far only a research project, Lumilo uses mixed-reality smart glasses to 
enable teachers to access a student’s real-time ITS data simply by looking 
at the student. In other words, Lumilo enables teachers to take advantage 
of ITS-driven adaptive learning and analytics, at the same time that they 
observe and engage with their classrooms as they would in a world 
without computers. 

The tool emerged from research that suggested teachers using ITS 
“wanted to be able to instantly see when a student is “stuck” (even if that 
student is not raising her/his hand), to instantly detect when a student is 
off-task or otherwise misusing the software, and to be able to see 
students’ step-by-step reasoning, unfolding in real-time.” However, 
while typical ITS might provide much of this information, they are not 
capable of registering or highlighting the many subtle cues exhibited by 
students that experienced teachers pick up on and use all the time. 

Accordingly, the Lumilo transparent smart glasses superimpose real- 
time indicators of students’ behavioral and learning states on top of the 
teachers’ view of their classroom (in other words it functions as an 
Augmented Reality system, which we discuss in more detail later). As the 
teacher looks around the classroom, observing their students’ TTS? 
activities, summary information appears, floating above each student’s 
head. Looking at a particular student, and clicking a handheld clicker ot 
making a specific hand-gesture, brings up a live feed of the student’s 
screen or more detailed information (such as the number of errors they 


have made or the number of hints they have requested). By combining 


3% Holstein, K., McLaren, B.M., and Aleven, V. (2018). “Student learning benefits of a mixed-reality teacher 
awareness tool in ai-enhanced classrooms.” In Artificial Intelligence in Education, ed. Rosé, C. P., et al. 
https://doi.org/10.1007/978-3-319-93843-1_12 

27 Holstein, K., et al. (2018). “The classroom as a dashboard: Co-designing wearable cognitive augmentation 
for K-12 teachers.” In Proceedings of the 8th International Conference on Learning Analytics and Knowledge—LAK 18. 
https://doi.org/10.1145/3170358.3170377 

308 Lumilo has been researched with an ITS, called Lynette, which has been designed to teach linear equations. 
Lynnette adaptively selects pathways using Bayesian knowledge tracing, and provides step-by-step guidance 
and feedback. See Aleven V., et al. (2016). “Example-tracing tutors: Intelligent tutor development for non- 
programmers.” International Journal of Artificial Intelligence in Education 26(1): 224-269. 
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these two types of data, ITS data and teacher observations, the Lumilo 
researchers aim to enable teachers to intervene appropriately with their 


students, as and when they decide. 


Language Learning: Babbel and Duolingo 


Another application of AI in education, which has recently seen 
considerable growth, is language learning. However, to digress briefly, 
potentially the most transformative application of natural language 
processing in recent times was the 2017 introduction of the Google Pixel 
Buds,” which emerged from the research that we discussed earlier into 
statistical approaches to natural language processing. The Pixel Buds’ 
algorithms, although far from perfect, are capable of translating between 
two spoken languages in real time, enabling two speakers who do not 
share a language to have a proper conversation—finally making real 
some long-anticipated science fiction gadgets. Remember the universal 


translator’ from Star Trek, or the Babel fish*"' from The Hitchhiker’s Guide 
to the Galaxy? 

Nonetheless, currently there remain good reasons for learning another 
language.*” Two of the most prominent Al-driven language learning 
commercial products, although there is little evidence that any are used 


extensively in formal educational settings, are Babbel and Duolingo (we 


3 https://www.blog.google/products/pixel/pixel-buds 

310 http://www.startrek.com/database_article/universal-translator 

311 http://hitchhikers.wikia.com/wiki/Babel_Fish 

312 Which of course calls into question whether language learning might become as quaint as massive 
memorization? The CCR will be tracking this issue closely. At present its recommendation is that language 
acquisition matters for three reasons: First, communication can be superseded by translation technology for 
conversational applications, though perhaps not for conversations that require fluency. Next is cultural 
understanding, which may be taught via other mechanisms, and finally, cognitive benefits, which are unclear 
through research. Will this become as indefensible as memorization after the invention of the alphabet? While 
this situation unfolds, CCR’s recommendation is that, given the sensitive period of language acquisition, one 
can easily set the foundation for multiple language acquisition through basic exposure to multiple languages. It 
was suggested that two languages of close linguistic distance could be mastered early (for instance for native 
English speakers: English at both for the first 2 years + Spanish or French by age 2-3; both Indo-European, 
but one Germanic and one Latin-based), and that maximum gains would come from a third that is very 
distant linguistically from a different linguistic family, and script-wise (for example, Mandarin or Arabic) 
taught by age 7. 

The key issue of time can be helped by technology as described herein. 
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could easily have chosen the similar Memrise" Rosetta Stone,** 


Mondly,*”” or many others). 


Our first example is Babbel,’”® 


‘which has been using Al-driven speech 
recognition (along with typical ITS personalization algorithms) to 
support language learning for about a decade. Their approach involves 
comparing student-spoken words with samples of speech recorded by 
native speaking course editors, and providing immediate feedback to help 
the student improve their pronunciation. There are two main steps: 
recognizing words and evaluating pronunciation, both of which can be 
challenging. To recognize words, the system first has to detect when the 
user starts and stops speaking, which in a typical environment means 
filtering out the ambient noises (such as other people speaking in the 
background or an airplane flying overhead). The words are then 
compared with the database of speech samples, first to recognize the 
word and then to check its pronunciation, taking into account that 
different people (male/female, young/old) have quite different voices 
(they speak at different frequencies and tempos). 

With Duolingo,”"’ which also uses speech recognition, we will focus 
on their use of ITS-style personalization. Duolingo’s approach draws on 
two principles that are well established in the learning sciences: the 


318 


spacing effect ' (we remember things more effectively if we use spaced 


practice, short study periods spread out over time, rather than massed 


319 


practice, or cramming) and the lag effect” (we learn even better if the 


spacing between practices gradually increases). Accordingly, an 


313 https://www.memrise.com 

314 https://www.rosettastone.co.uk 

315 https://app.mondly.com 

316 https://www.babbel.com 

317 https://www.duolingo.com 

318 Ausubel, D.P., and Youssef, M. (1965). “The effect of spaced repetition on meaningful retention.” The 
Journal of General Psychology 73: 147-50. https://doi.org/10.1080/00221309.1965.9711263 

3 Melton, A.W. (1970). “The situation with respect to the spacing of repetitions and memory.” Journal of 
Verbal Learning and Verbal Behavior 9 (50): 596-606. 

32 Duolingo is not unique in doing this but they it is notable in having conducted various studies to optimise 


the approach. 
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algorithm based on the Leitner Box method,” in which flashcards 
answered incorrectly remain at the front of a containing box to be 
encountered again after only a short interval, while those answered 
correctly are sent to the back of the box thus delaying when they will be 
seen again) predicts the best time to deliver a word to the student for 
practice. It does so by inferring the probability that the student will 
correctly recall the word as a function of the lag time since the word was 
last practiced and the half-life of the word (the strength of the word in 
the learner’s long-term memory). This is based on a student model that 
incorporates trace data of the learner’s previous learning experiences (the 
number of times a student has seen the word, the number of times it was 
correctly recalled, and the number of times incorrect), A 2012 
independent study found that students using Duolingo improved their 
Spanish language abilities by the equivalent of one college semester of 
standard language classes (however, the study did not compare Duolingo 
with standard language classes or with any similar product, and no effect 


sizes are given). 
Chatbots: Ada and Freudbot 


The first computer program that appeared able to converse in natural 
language, the precursor to AI chatbots, was ELIZA (which we describe 
in appendix 2). Now, after fifty years of development, chatbots are 
becoming mainstream,” with the launch of digital assistants from tech’s 
Big Five: Amazon (Alexa), Apple (Siri), Facebook (Messenger), Google 
(Assistant), and Microsoft (Cortana). Nonetheless, progress has not 


always been straightforward (remember the racist rants tweeted by 


321 Leitner, S. (1995). So Lert Man Lernen: Angewandte Lernpsychologie—Ein Weg Zum Erfolg. Herder. 

322 E.g., Dale, R. (2016). “The return of the chatbots.” Natural Language Engineering 22 (5): 811-817, and 
“Everything you ever wanted to know about chatbots (but were afraid to ask).” 

https:/ /wwwjisc.ac.uk/blog/everything-you-ever-wanted-to-know-about-chatbots-but-were-afraid-to-ask-08- 
oct-2018 

323 https://messenger.fb.com 
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Microsoft’s chatbot Tay?),”* and no chatbot has yet convincingly passed 
the Turing Test (if a human cannot decide whether a computer is human 
ot a computer, the computer is said to have passed the Turing Test)” 
Having said that, recently the Google Duplex chatbot was presented 
making a restaurant reservation and booking a hairdresser appointment 
(however, the demonstration was clearly carefully orchestrated and more 
than a little controversial). Nonetheless, today, one chatbots-as-a- 
setvice company alone’ claims that more than 300,000 bots have been 
created using its toolsets, and this is just one of many such platforms.” 
In general, chatbots are designed to respond automatically to 
messages (SMS texts, website chats, social messaging posts, and voice), 
using rules and keywords to select from pre-programmed scripted 
responses (as with ELIZA and most current simple bots), or adaptive 
machine learning algorithms to generate unique responses (as with the 
mote sophisticated bots such as Siri, Duplex, and Tay). Chatbots are 
quickly becoming ubiquitous, for everything from booking a flight,” to 


: 330 Zio | : . . : 
ordering food,” as a doctor,” or a financial adviser,” in recruitment,” 


324 Wolf, M.J., Miller, K., and Grodzinsky, F.S. (2017). “Why we should have seen that coming: Comments on 
Microsoft’s tay ‘experiment,’ and wider implications.” SIGCAS Comput. Soc. 47 (3) 54-64. 
https://doi.org/10.1145/3144592.3144598, 

325 The Turing Test, or more correctly the Imitation Game, was devised by Alan Turing (who is regarded by 
many as the father of both modern computing and artificial intelligence), to determine whether we might 
consider a computer intelligent: “I believe that in about fifty years’ time it will be possible to program 
computers ... to make them play the imitation game so well that an average interrogator will not have more 
that 70 percent chance of making the right identification after five minutes of questioning.” Turing, A. (1950). 
“Computing machinery and intelligence.” Mind 59 (236): 433-460. 

326 See both https://www.extremetech.com/computing/269030-did-google-duplexs-ai-demonstration-just- 
pass-the-turing-test and https://www.extremetech.com/computing/269497-did-google-fake-its-google- 
duplex-ai-demo 

327 https://home.pandorabots.com 

328 https: / /www.techworld.com/picture-gallery/apps-wearables/platforms-for-developers-build-chatbots- 
3639106 

329 https://bb.klm.com/en 

330 https://www.tacobell.com/Tacobot 

331 https://www.your.md 

332 https://www.tbs.com/rbs/news/201 6/03/rbs-installs-advanced-human-ai-to-help-staff-answer-customer- 
que.html 

333 https://hiremya.com 
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and accounting,” to help with shopping,” as a personal companion, 
and to support young people who are suffering from anxiety.” 

Chatbots are also increasingly being used in educational contexts for a 
variety of purposes. For example, students making initial enquiries about 
courses may find themselves conversing with a bot whose job it is to 


8 In some 


direct them to the information that they want.” 
configurations,” they can also provide ongoing student support and 
guidance, in academic services, accommodation, facilities, examinations, 
IT, health and more.’ And in some situations, they can also be used to 
directly support learning—indeed, the DBTS that we met earlier 
(including AutoTutor and Watson Tutor), may be considered special 


cases of educational chatbots). For example, chatbots might provide 


+! while already 


feedback to support student reflection and self-efficacy,” 
some language learning apps use chatbots for embarrassment-free 
speaking practice in simulated real-life situations.” 

However this is not to suggest that chatbots are an educational silver 
bullet. For example, a student’s “willingness to communicate in a foreign 
language... seems to decline rapidly over time as students lose interest in 


chatbots as language partners compared to human learning partners. This 


3 https://www.sage.com/en-gb/products/pegg 

35 https://bots.kik.com/#/vspink 

336 https://www.pandorabots.com/mitsuku 

37 Fitzpatrick, K.K., Darcy, A., and Vierhile, M. (2017). “Delivering cognitive behavior therapy to young 
adults with symptoms of depression and anxiety using a fully automated conversational agent (woebot): A 
randomized controlled trial.” JMIR Mental Health 4. https://doi.org/10.2196/mental.7785 

338 https://www.vittualspirits.com/chatbot-for-university.aspx 

3° https://www.slu.edu/alexa/index.php 

30 E.g., Deakin University uses IBM Watson to run a student services support chatbot: 
http://www.deakin.edu.au/about-deakin/media-releases /articles /ibm-watson-helps-deakin-drive-the-digital- 
frontier, while the Open University of Hong Kong launched The i-Counseling System. 

https:/ /library.educause,edu/resources/2012/5/case-study-9-the-open-university-of-hong-kong-the- 
icounseling-system 

31 Lundqvist, K.O., Pursey, G., and Williams, S. (2013). “Design and implementation of conversational agents 
for harvesting feedback in elearning systems.” In European Conference on Technology Enhanced Learning, 617-618. 


32 http://bots.duolingo.com 
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could happen because of a simple novelty effect or simply the weaker 
value of chatbots compared to human assistants.’>? 

Typical education chatbots (in addition to the DBTS discussed earlier) 
are Ada and Freudbot. Ada™* has been developed by a UK community 
college, using IBM’s Watson Conversation platform. In short, Ada 
demonstrates how education chatbots have been implemented using 
limited resources and Al-as-a-service technologies. Available on multiple 
devices (desktop, mobile, and kiosk), and named after the computer 
pioneer Ada Lovelace, Ada is able to respond to a spectrum of student 
enquiries, delivering personalized and contextualized responses that draw 
on data such as the student’s courses, their progress, their goals and their 
individual targets. Ada is already able to respond to questions about the 
library, student services, finance, accommodation, transport, careers, and 
examinations—and it learns more with every interaction. For example, a 
student might ask about their lessons that morning, or where tomortow’s 
exam is happening, or what mark they achieved in a recent assignment. 
Meanwhile, a teacher can ask for a list of professional development 
workshops they have recently attended, or about a specific student’s 
academic performance. 

An earlier chatbot (and in effect a primitive DBTS) called Freudbot*® 
engaged students in a conversation about an educational topic (rather 
than being designed to provide students with information about their 
institution and studies, as in the case of Ada). It took on the tole and 
persona of the psychoanalyst Sigmund Freud, chatting in the first person 
with introductory psychology students about his theories and life. 
Freudbot was developed before the availability of easily accessed 
machine learning techniques, and so used rules and recognized keywords 


to select from pre-programmed scripted responses, drawing on a 


*8 Winkler, R. and Soellner, M. (2018). “Unleashing the potential of chatbots in education; A state-of-the-art 
analysis.” Academy of Management Proceedings (1): 1-17. https://doi.org/10.5465/AMBPP.2018.15903abstract 
34 http://www.aftabhussain.com/ada.html 

%5 Heller, B., et al. (2005). “Freudbot: An investigation of chatbot technology in distance education.” In 
EdMedia: World Conference on Educational Media and Technology, 3913-3918. 
https://pdfs.semanticscholar.org/ba80/d43699062892440£7e9adbGaea8e3cal ddfe.pdf 
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university resource comprising explanations of Freudian terms and 
concepts together with an open-source biography. And when a student’s 
question or response was outside its rule-base, Freudbot would default to 
a backstop strategy, such as asking for clarification or suggesting a new 
topic for discussion, always with the aim of leading the user back to the 


core discussion of Freudian topics. 


Augmented and Virtual Reality 


Virtual reality (VR) and augmented reality (AR) are two innovations that 
many have been trying to apply in educational contexts” (for example, 
Google has developed for educational contexts more than a 1000 VR and 
AR Expeditions). VR provides an immersion experience that shuts out 
the physical world, enabling users who are wearing appropriate VR 
goggles to be transported into a vast range of real world or imagined 
environments, such as the International Space Station, an operating 
theatre,” or Hogwarts castle.” AR, on the other hand, as we saw earlier 
with Lumilo, overlays computer-generated images on top of the uset’s 
view of the real world (much like a fighter pilot’s heads up display), ” 
when seen through a smartphone or other similar device—the aim being 
to enhance or mediate the uset’s view of reality. For example, when a 
smartphone’s camera is pointed at a mountain range, the names of the 
mountains and their elevations might be supetimposed,™' while pointing 
at a particular QR code might reveal a 3D human heart that can be 
explored in detail,” and in a particular street location a Pokémon 
character might be found waiting to be caught.*? Although not 


traditionally thought of as AI technologies, we have mentioned VR and 


36 E.g., http://www.classvr.com 

347 https://edu.google.com/expeditions 

348 http://ossovr.com 

+ https://www.pottermore.com/news/new-expanded-fantastic-beasts-and-where-to-find-them-vr- 
experience-announced 

35° https://www.youtube.com/watch?v=Ay6g66FbkmQ 

31 https://www.peakfinder.org 

32 https://medmovie.com/augmented-reality-heart 


353 https://www.pokemongo.com/en-gb 
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AR here because both are frequently combined with AI machine 
learning, image recognition and natural language processing, with the aim 
of further enhancing the user experience.’ 

While the thought of thirty students all wearing goggles and immersed 
in another world might strike fear in the heart of a classtoom teacher, 
and while “VR does not intrinsically make evety experience better in 
terms of motivation and learning,” used judiciously, both VR and AR 
do have potential to become useful tools in the educatot’s toolbox. To 
give a few brief examples: VR has been used effectively to give breast 
cancer patients an anxiety-relieving virtual experience of the radiotherapy 
process, tailored to each individual patient,” VR simulations have been 
used extensively to provide training for neurosurgical residents on a 
variety of neurosurgical procedures,” and to enable students to interact 
directly with historical characters, while a VR classroom has been used 
to provide trainee teachers with an “absorbing, realistic and interactive 
virtual classroom, allowing them to engage in realistic interactions with 
virtual students.”*” Researchers have also proposed the use of VR to 
enhance student experiences in immersive simulations such as 
EcoMUVE,™” an immersive multi-user virtual environment and 
associated inquiry-based curriculum developed at Harvard University. 


EcoMUVE was designed to enable school students to leatn about 


4 Eg. https://www.apple.com/uk/ios/augmented-reality, 
https://www.samsung.com/global/galaxy/galaxy-s9 /augmented-teality and https://ametroslearning.com 
5 Dede, C., et al. (2017). “Virtual reality as an immersive medium for authentic simulations.” 
https://doi.org/10.1007/978-981-10-5490-7_8 

356 Jimenez, Y.A., et al. (2018). “Patient education using virtual reality increases knowledge and positive 
experience for breast cancer patients undergoing radiation therapy.” Supportive Care in Cancer 26 (8): 2879-88. 
https://doi.org/10.1007/s00520-018-4114-4 

37 McGuire, L.S. and Alaraj, A. (2018). “Competency assessment in virtual reality-based simulation in 
neurosurgical training.” In Comprehensive Healthcare Simulation: Neurosurgery. Springer. 153-157. 

*8 Baierle, LL.F., Gluz J.C. (2018) “Programming intelligent embodied pedagogical agents to teach the 
beginnings of industrial revolution.” In Nkambou, R., Azevedo, R., Vassileva, J. (eds.) Intelligent Tutoring 
Systems. Lecture Notes in Computer Science 10858. Springer. https://doi.org/10.1007/978-3-319-91464-0_1 

3 Stavroulia, K.E., et al. (2018). “Designing a virtual environment for teacher training: Enhancing presence 
and empathy.” In Proceedings of Computer Graphics International 2018 on CGI 2018. ACM Press. 
https://doi.org/10.1145/3208159.3208177 

36 http://ecolearn.gse.harvard.edu 
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ecosystems by playing the role of a scientist, exploring and collecting data 
in a virtual ecosystem in order to answer research questions. Although a 
VR interface might make some tasks more difficult, the researchers 
suggest that it has the potential to make the simulation mote realistic by 
increasing the students’ feeling of being present in the simulated 
environment, which in turn is likely to enhance transfer of the learning 
from the virtual to the real world.” 

AR, on the other hand, has been used to enable students to explore 
and manipulate three-dimensional models of organic molecules in order 


362 


to enhance their understanding of chemistry,” to help primary school 


students learn about history,’ and in an AR-enabled digital games-based 
learning environment to support students’ reading comprehension.” 
These few examples only touch the surface of the research that has 
investigated VR and AR in education.” 


Learning Network Orchestrators: 


Third Space Learning and Smart Learning Partner 


366 


Learning network orchestrators (LNO),” are tools or approaches that 


enable and support networks of people engaged in learning (students and 
their peers, students and their teachers, or students and people from 
industry). LNOs typically match participants based on their availability, 


the subject domain, and the participants’ varied expertise, and can 


361 See Dede et al. (2017), in which they discuss in depth the potential and limitations of VR for simulated 
environments, and suggest some useful principles for effective implementation. 

362 Behmke, D., et al. (2018) “Augmented reality chemistry: Transforming 2-D molecular representations into 
interactive 3-D structures.” Proceedings of the Interdisciplinary STEM Teaching and Learning Conference 2(1). 
https://doi.org/10.20429/stem.2018.020103 

363 Efstathiou, I., Kyza, E.A., and Georgiou, Y. (2018). “An inquiry-based augmented reality mobile learning 
approach to fostering primary school students’ historical reasoning in non-formal settings.” Interactive Learning 
Environments 26 (1): 22—41. https://doi.org/10.1080/10494820.2016.1276076 

364 Tobar-Murioz, H., Baldiris, S. and Fabregat, R. (2017) ” Augmented reality game-based learning: Enriching 
students’ experience during reading comprehension activities.” Journal of Educational Computing Research 55 (7). 
http://journals.sagepub.com/doi/10.1177/0735633116689789 

365 Radu, J. (2014) “Augmented reality in education: A meta-review and cross-media analysis.” Personal and 
Ubiquitous Computing 18 (6): 1533-1543. 

366 E.g., Nepris (https://www.nepris.com) and Educurious (https://educurious.org), which both support 
schools to connect with experts from around the world, to bring an industry perspective into the classroom. 


Possibilities include interactive question and answer sessions, virtual field trips, and project mentorships. 
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facilitate coordination and cooperation between them: “participants can 
interact with one another, share their learning experiences, build 
relationships, share advice, give reviews, collaborate, co-create and 
more.” %7 AI techniques are slowly being introduced to LNO products, 
to automate much of this orchestration, and opening up network 
possibilities previously unachievable. 

For example, in a novel approach, Third Space Learning connects UK 
ptimary school children who are at risk of failure in mathematics with 
mathematics tutors in India and Sri Lanka. The system supports 
individual tutoring, with tutors and students communicating with each 
other by means of a secure online virtual classroom that has two-way 
audio and a shared interactive whiteboard. AI is being introduced to 
automatically monitor every session, thousands of hours of teaching and 
learning every week, which generates huge quantities of data. Algorithms 
then aim to guide the tutor with real-time feedback, ensuring that 
teaching broadly follows an outline script that adopts well-supported 
learning sciences principles, identifying where students have 
misconceptions not picked up by the tutor (by comparing individual and 
overall student models), and empowering the tutors to constantly 
improve their teaching skills. 

The Smart Learning Partner,’ on the other hand, uses much simpler 
AI technologies to put students more in control of their own learning. It 
is the result of a collaboration between Beijing Normal University’s 
Advanced Innovation Center for Future Education and Tongzhou 
district of Beijing, in China. A key component of the Smart Learning 
Partner system is an Al-driven platform that enables students to connect 
with a human tutor via their mobile phones. The platform uses AI 
somewhat like a dating app—except it matches students and tutors 
according to student queries and tutor areas of expertise, together with 
the tutor’s availability and ratings given to them by other students whom 
they have already tutored. The student uses the app to search for a tutor, 


%7 Holmes, W., et al. Technology-Enhanced Personalised Learning. 
38 http://slp.bnu.edu.cn (Note: Only accessible to students and faculty who have an account.) 


150 Artificial Intelligence in Education 


to ask what they want to know about any school topic, and they then 
receive twenty minutes of one-to-one online tuition (sharing audio and 


screens only). 
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What Else is Possible? 


Much of the ATED that we have discussed so far involves the application 
of AI techniques to mainstream learning approaches, and tends to reflect 
(or automate) existing educational assumptions and practices. In addition, 
although we have seen some notable exceptions, much AIED has been 
designed (whether intentionally or not) to supplant teachers or to reduce 


3 and not to assist them to teach more 


them to a functional role, 
effectively. This approach, while potentially useful in contexts where 
teachers are few and far between, clearly undervalues teachers’ unique 
skills and experiences, as well as learners’ needs for social learning and 
guidance. However, instead of just automating the teaching of students 
sat at computers, conceivably AI might help open up teaching and 
learning possibilities that are otherwise difficult to achieve, that challenge 
existing pedagogies, or that help teachers to be mote effective. Here we 
will speculate on some possibilities, some of which have been 
foreshadowed by the AIED tools we have already discussed, while others 
ate both novel and complex to achieve, and all of which raise interesting 
social questions. We begin with AI to support collaborative learning, then 
Al-driven student forum monitoring, AI to support continuous 
assessment, AI learning companions for students, and AI teaching 
assistants for teachers, before concluding with AIED as a research tool 
to further the learning sciences (i.e. in order to help us better understand 


. 7 
learning).°” 


Collaborative Learning 


Collaborative learning, where students work together to solve problems, 


is well known to be able to lead to better learning outcomes, but effective 


3 Worryingly, one of the developers we have mentioned has suggested that the sophistication of their AIED 
means that teachers only need to play an auxiliary role, working like fast-food chefs (“KFC-like”) to strictly 
regulated scripts. 

370 One intriguing use of AI in education that we will not consider in detail, because its efficacy has not yet 
been demonstrated, but that should still be acknowledged is the automatic generation of quiz questions 
(https://mt.clevere.st and https://learningtools.donjohnston.com/product/quizbot). 
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collaboration between learners can be difficult to achieve.’”' ATED offers 
various possibilities. To begin with, an AIED tool could automatically 
suggest groups of students best suited for particular collaborative tasks, 
drawing on and making intelligent connections between individual 
student models (each of which comprises knowledge about the student’s 
previous learning experiences and achievements, what the student is 
learning in other classrooms, their personalities, and more).”” Having 
elicited the teachers’ requirements, the tool might also suggest groups of 
mixed or similar-ability students, or groups designed to give particular 
students opportunities to take on leadership roles, or groups that avoid 
personality or temperament clashes, and so on, all the while enabling the 
teacher to quickly and easily override any of the tool’s suggestions (which 
the AI will learn from, for next time). An AIED tool might also take on 
the role of expert facilitator or moderator, monitoring student 
collaborative activities, recognizing when students are having trouble 
understanding shared concepts, and then providing targeted support. 
Alternatively, the AIED might involve a virtual agent that actively 
contributes to the group discussions (acting as a virtual peer or a 
teachable agent), or that makes dynamic connections (either with 
discussions being held by other groups in the same classroom, or with 
relevant materials drawn from the semantic web). In fact, some research 
into AI to support collaborative learning has been undertaken,” but 
there are many technical issues to overcome before it becomes possible 


in real classrooms. 


371 Luckin, R., et al. (2017). Solved! Making the Case for Collaborative Problem-Solving, Nesta. 
https://www.nesta.org.uk/report/solved-making-the-case-for-collaborative-problem-solving/ 

372 The Universitat Politécnica de Valéncia have been researching just such a system: Alberola, J.M., del Val, 
E., Sanchez-Anguix, V., Palomares, A., and Teruel, M.D. (2016). “An artificial intelligence tool for 
heterogeneous team formation in the classroom.” Knowledge-Based Systems 101: 1-14. 
https://doi.org/10.1016/j.knosys.2016.02.010 

33 E.g., Diziol, D., et al. (2010). “Using intelligent tutor technology to implement adaptive support for student 
collaboration.” Educational Psychology Review 22 (1): 89-102. https://doi.org/10.1007/s10648-009-9116-9 and 
Spikol, D., et al. (2016). “Exploring the interplay between human and machine annotated multimodal learning 
analytics in hands-on stem activities.” In Proceedings of the Sixth International Conference on Learning Analytics & 
Knowledge. 522-523. 
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Student Forum Monitoring 


Increasingly, students of all ages are participating in online education, 
which usually involves the use of discussion forums. Students might post 
to forums in response to given tasks or to engage in collaborative 
learning opportunities, or they might want to contact their tutors to 
clarify course requirements or to ask about course materials. Accordingly, 
especially when there are large cohorts of students (as can be typical of 
some distance universities and MOOCs), these online forums can 
generate massive numbers of forum posts, all of which must be 
monitored, moderated, and addressed. However, as the number of forum 
posts increases, this becomes at best an inefficient use of a tutor’s time 
(dealing with repetitive and minor practical issues) and at worst an 
increasingly impossible task. It also makes it difficult for students to keep 
up to date with other student posts that might connect to their interests. 

AIED might help in a number of ways (again, some research has 
already been conducted in this area)*”*—in particular by helping the 
teachers/tutors to be better able to support their students. First, an 
AIED tool might triage the forum posts, identifying those that can be 
dealt with automatically (perhaps practical questions around course dates, 
such as “When do I need to submit...?’’), and those that require a 
response from a human tutor (such as those discussing more in-depth 
core subject issues). The simple posts, the ones that the ATED is capable 
of handling, would receive immediate automatic responses, relieving the 
human tutors of much repetitive work while enabling the students to 
move on quickly to more substantive work. Other posts would 
automatically be referred up to a human tutor, to ensure that students 
receive high quality, appropriate responses whatever the nature of their 
posting. 

Taking this a step further, the more demanding posts (of which there 
still might be many) would be further analyzed, the aim being to identify 


374 Goel, A.K., and Joyner, D.A. (2017). “Using AI to teach AI: Lessons from an online AI class.” AI Magazine 
38(2): 48. https://doi.org/10.1609/aimag.v38i2.2732 
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and aggregate similar posts or posts that raise overlapping issues (in a 
coutse with a thousand students, it is unlikely that there will be a 
thousand unique responses to a single course activity, but rather a much 
smaller number of closely related posts). A human tutor would then write 
a response to the much smaller number of aggregated posts, which in 
turn would be issued to all of the original posters. Although this is 
unlikely to be as good as replying to each individual student, it would 
clearly be better than the students receiving no responses at all—which, 
in a large online course, can all too often be the case. Another approach 
that might also help in student forums is for the AIED to interpret and 
make dynamic connections between posts, informing tutors when 
particular issues have been raised (e.g, known and unknown 
misconceptions), for them to address, or informing students about other 
posts that they might find interesting. 

Finally, the AIED might also use sentiment analysis AI techniques to 
identify posts that reveal negative or non-productive student emotional 
states (perhaps a student is overly challenged, or likely to drop out of the 
course, ot possibly suffering from mental health issues), posts that are 
unacceptable (perhaps because they include racist, misogynist or 
gratuitously aggressive comments), or posts that suggest topic drift (the 
tendency for forum posts to drift from the original intent). Any such 
posts (which, because of the overall number of posts, can be easy for 
humans to miss) would be referred up to a human tutor, so that the tutor 
can respond quickly, appropriately and effectively (perhaps by calling the 
student by phone, rather than depending on a digital intervention). 
Together, these various techniques might also enable tutors to be kept 
well informed of student opinions, collective worries, or recurrent 


themes that emerge from the forums. 
Continuous Assessment 


Psychologists and educators know that it is wrong to make decisions 
based upon a single test score and that decisions should reflect a 


balanced, complete understanding of each child. Numbers and scores can 
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be very misleading if we don’t consider the whole picture, something that 
means using both a qualitative and quantitative approach.’” 

Although there is little evidence for their validity, reliability or 
accutacy, high-stakes examinations are core to educational systems 
around the world.’ Perhaps this is because that is how it has always 
been, perhaps because they efficiently rank students, perhaps because no 
practical, cost effective at scale, alternative has ever been devised, or 
perhaps because those who run the systems are typically those who were 
most successful at exams (and do not emotionally resonate with the need 
for change). Whatever the reason, with high-stakes examinations in place, 
schools and universities all too often end up teaching to the test, 
prioritizing routine cognitive skills and knowledge acquisition over in- 
depth understanding and authentic application. In other words, the 
examinations, rather than the needs of students or wider society, 
determine what is taught and learned. Meanwhile, ironically, AI 
technologies are automating exactly the type of knowledge that 
examinations mostly assess: “There’s lots of elements of human 
intelligence that cannot be automated but the bit that we’ve tended to 
value, that relates to academic exam success, is one of the bits that we’ve 


377 
managed to automate.” 


In any case, stop-and-test examinations 
(standardized, unseen tests that are at set points in the learning schedule, 
thus potentially interrupting the learning) are not able to rigorously 
evaluate a student’s understanding of all that has been learned—at best 
they can only provide a snapshot of fragments of what has been studied 
over the duration of a course. Last, but not least, students of all ages can 
sometimes suffer from serious exam anxiety, which can easily negatively 
impact on the student’s success in a typical three-hour end of course 


examination (further clouding their accuracy and trustworthiness). 


*> Gunzelmann, B.G. (2005). “Toxic testing: It’s time to reflect upon our current testing practices.” 
Educational Horizons 83 (3): 214. 

376 https://curriculumredesign.org/wp-content/uploads/Evolving-Assessments-for-the-21 st-Century-Report- 
Feb-15-Final-by-CCR-ARC.pdf 

37 Rose Luckin quoted in https://www jisc.ac.uk/news /the-ai-revolution-is-here-17-aug-2018 
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Nonetheless, most AIED research in this area has been unambitious. 
It has focused on improving existing examination systems (developing 
Al-driven techniques to authenticate the identity of students taking 
exams online), rather than challenging the underlying principles. 
However, as we have seen, typical ITS and other AIED tools already and 
constantly monitor student progress to provide targeted feedback and to 
assess whether the student has achieved mastery of the topic in question. 
Similar information could be captured by AIED tools designed to 
support collaborative learning, while intelligent essay assessment tools 
can also make inferences about a student’s understanding. All of this 
information and more might be collated throughout a student’s time in 
formal educational settings (the learning sciences have long understood 
the value of students engaging with constructive assessment activities), 
together with information about the student’s engagement with non- 
formal learning (such as learning a musical instrument, or a craft or other 
skills) and informal learning (such as language learning or enculturation 
by means of learning from experience or active participation), to help 
create a picture of the whole learner. In other words, the AlI-driven 
assessment would happen in the background, all of the time—making it 
next to impossible for students to cheat or subvert the system’s intention 
(as can be the case when wealthier students employ personal tutors),”” ot 
take the test as many times as necessary until they achieve a good-enough 
score. 

This more detailed and nuanced information about an individual 
student might then be represented (and perhaps visualized in dynamic 
graphics) in an Al-driven e-portfolio,™ an intelligent personal resumé (in 


fact, an extended open student model). This e-portfolio could perhaps be 


38 For example, http://tesla-project.eu 

3 Luckin, R. (2017). “Towards artificial intelligence-based assessment systems.” Nature Human Behaviour 1. 
https://doi.org/10.1038/s41562-016-0028 

38° Per one of the authors’ US patent numbers 9,262,640 and 9,582,567, which also protect privacy and 
security. 
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underwritten and authenticated by blockchain technologies”! as used by 
virtual currencies such as Bitcoin (essentially open, distributed ledgers, 
hosted simultaneously by millions of computers across the internet and 
linked using cryptography, that can share data in a verifiable, 
incorruptible, and accessible way). In this way, students would have a 
robust, accredited, in-depth record of all their learning experiences and 
achievements, far more detailed and useful than a collection of 
certificates. Parts or all of this smart resumé they might share when 
applying for admission to another course or for a new job, while 
retaining full control of their academic persona and data. From a learnet’s 
perspective, an additional benefit is that continuous assessment can act as 
a moving average, a fluid-like shock absorber that evens out the blips of 
bad days and disadvantageous personal situations (it simply does not 
sense that a young person’s academic outcomes and future life can be 
determined by difficulties at home that coincide with the day of an 
important exam). 

In short, although the constant monitoring of student behaviors and 
achievements raises significant and far-reaching ethical questions that 
must first be properly investigated and addressed, it is conceivable that 
stop-and-test examinations could soon be entirely removed from our 


educational systems and relegated to a more primitive past. 


AI Learning Companions 


The smart resumés that we have just proposed could also play a role in a 
much larger AIED possibility: Al-driven lifelong students’ learning 
companions. As we have seen, the desire for every student to have 
their own personalized tutor is what first inspired the development of 
ITS, but what about taking this to its logical conclusion? AI has the 
potential to provide every student with their very own personalized 


learning companion, operating sometimes as a learning partner, other 


381 Sharples, M. and Domingue, J. (2016). “The blockchain and kudos: A distributed system for educational 
record, reputation and reward.” In European Conference on Technology Enhanced Learning, Springer. 490-496. 
38 The University of Southern California have been researching just such an application over many years: 


http:/ /ict.usc.edu/ prototypes /personal-assistant-for-life-long-learning-pal3 
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times as a guide through the mass of available learning opportunities, and 
sometimes as an instructor, all the time recording the student’s interests 
and progress in their blockchain-protected, smart resumé. The arrival and 
rapid developments of Siri, Cortana, Google Home and Alexa, suggest 


that this possibility is tantalizingly close. 


In many countries, 
smartphones with extraordinary processing power and always-on internet 
access ate more than common. It would not necessarily be a big technical 
step to leverage these capabilities, to create an Al-driven smartphone 
learning companion that could accompany and support individual 
learners throughout their studies, from kindergarten to old age. 

Such a learning companion brings many possibilities. Once the 
student has decided on a particular topic of interest, it might provide 
some instructional activities, monitor the student’s progress, remind 
them when a task needs to be completed, and offer targeted feedback 
and guidance—all on their speech-driven smartphone (and available on 
all their other devices). In other words, it might function as what we have 
called an ITS+. 

But a learning companion would also operate at a higher and more 
strategic level. Building on the student’s individual interests and life goals, 
it could also help them decide what to learn, as well as where and how to 
do the learning (the companion might identify and connect with the 
learning opportunities that are available, both formal and informal, both 
on and off-line). It could then also guide the student along overarching 
long-term individualized learning pathways designed to help the student 
address their emerging personal life-goals, connecting their learning 
interests and achievements, while reminding them of and encouraging 
them to reflect on and perhaps develop their long-term learning aims. 
The learning companion’ might suggest learning opportunities that 


385 


focus on some so-called 21st Century Skills,” and social-emotional 


383 https: //www.theatlantic.com/magazine/archive/2018/11/alexa-how-will-you-change-us/570844/ 
384 World Economic Forum. (2015). New Vision for Education: Unlocking the Potential of Technology. 
World Economic Forum. 

385 Trilling, B. and Fadel, C. (2012). 27st Century Skills: Learning for Life in Our Times. John Wiley & Sons. 
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learning.’ It could also potentially connect learners, in the same 
classroom or from opposite sides of the world, depending on their 
shared interests and goals, helping them develop and work together in 
ptojects that prioritize both individual and collective achievements (and, 
in turn, helping to promote other critical skills in collaboration, 


teamwork, and intercultural awareness). 


AI Teaching Assistant 


As we have noted several times, most AIED technologies ate designed 
with the aim of relieving teachers of the grunt work of teaching (most 
often by automating time-consuming activities such as the marking of 
classroom or homework assignments). However, despite these best of 
intentions, many AIED technologies in effect take over teaching (they 
deliver personalized and adapted learning activities better than teachers), 
or at least they reduce teachers to a functional role (perhaps their job is to 
work to strictly regulated scripts, or to ensure that the technology is ready 
for the student to use). Nonetheless, as we and colleagues have written 
previously: Crucially we do not see a future in which AIED replaces 
teachers. What we do see is a future in which the role of the teacher 
continues to evolve and is eventually transformed; one where their time 
is used mote effectively and efficiently, and where their expertise is better 
deployed, leveraged, and augmented.” 

This might be more of an emotional plea than a coherent argument— 
but it assumes that teaching involves more than delivering’ knowledge, 
and that it is a fundamentally social process. From this perspective, a key 
tole for AI is supporting teachers to teach and support students. 

One way in which this might be achieved is by augmenting teachers’ 
expertise and skills with an AI teaching assistant, to complement and 
work with the students’ AI learning companion, that goes far beyond the 


useful but by comparison somewhat primitive teacher dashboards 


386 Fadel, C., Bialik, M., and Trilling, B. (2015). Fowr-Dimensional Education: The Competencies Learners Need to 
Succeed. Center for Curriculum Redesign. 
387 Luckin. R., et al. Intelligence Unleashed, 11. 
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featured in so much education technology. This would be a key way that 
AIED can support teachers to support students. Just such a possibility 
has been explored in the short narrative “A.I. is the New T.A in the 


Classroom,” 


which describes a possible classroom of the future in 
which the teacher is supported by a dedicated and personalized AI 
teaching assistant (AI TA). 

Many of the ideas we have suggested could play a role in this possible 
scenario (such as automatically setting up collaborative groupings of 
students, replacing stop-and-test examinations with Al-supported 
continuous assessment, and managing peer-marking and undertaking 
some automated marking). The AI TA could also automatically provide 
teaching and professional development resources (texts, images, videos, 
augmented-reality animations, links, network connections) that the 
teacher might choose to call upon to support their teaching. It could also 
monitor the students’ performance as they engage in their classroom 
activities, continuously updating their learner models, making 
connections with the domain models of topics being taught, and tracking 
progress over time. All of this information (together with data about each 
student from additional sources: assessments from other classes, informal 
learning achievements, and relevant medical or family information) could 
be readily available to the teacher, whenever the AI TA computes it 
might be useful or whenever the teacher calls for it. In this possible 
future, what and how to teach the students, and how best to support 
them, would remain the responsibility and prerogative of the teacher. 
The AI TA’s role would simply be to make the teacher’s job easier and 


more effective. 


AIED as a Research Tool to Further the Learning Sciences 


As has probably been noticed, each of AIED’s possible future uses are 
firmly rooted in existing AIED research and approaches. This is no less 


true of our final example, the use of AIED as a research tool to further 


388 Luckin, R., and Holmes, W. (2017). “A.I. is the new T.A. in the classroom.” How We Get To Next. 


https://howwegettonext.com /a-i-is-the-new-t-a-in-the-classroom-dedbe5b99e9e 
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the learning sciences. Implementing an educational practice in any 
technology means that the practice has to be both better understood and 
then systemized. As a consequence, the technology acts much like a 
virtual spotlight, highlighting issues that have existed for years but that 
have been hidden or overlooked (for example, around the most effective 
approaches to teaching). This is particularly true of the introduction of 
AI to education, which is beginning to throw an extraordinarily bright 
spotlight onto many learning sciences issues. However, while there have 
been notable developments in this area of AIED research, mostly it has 
been at a relatively theoretical level, such that their potential and 
implications remain somewhat unclear. 

In fact, AIED as a learning sciences research tool is often linked to a 
pair of other independent but overlapping academic fields that use 


statistical techniques drawn from big data research: learning analytics 


390 


and educational data mining.” While learning analytics involves “the 


measurement, collection, analysis and reporting of data about learners 


and their contexts, for purposes of understanding and optimizing 


22391 


learning and the environments in which it occurs; educational data 


392 


mining’ ~ “is concerned with gathering and analyzing data so as to 


understand, support and improve students’ learning.” One example, 
that avoids this distinction, and that has been shown to be effective, is 


389 Mayer-Schonberger, V. and Cukier, K. (2013). Big Data: A Revolution That Will Transform How We Live, Work 
and Think. John Murray. 

3% Readers who would like to learn more about the similarities and differences between learning analytics and 
educational data mining might be interested to read Benedict du Boulay and others, “What does the research 
say about how artificial intelligence and big data can close the achievement gap?” in Luckin, R. (ed.) (2018). 
Enhancing Learning and Teaching with Technology. Institute of Education Press, 316-27; or Siemens, G., and Baker, 
R.SJ.d.. (2012). “Learning analytics and educational data mining: Towards communication and collaboration.” 
In Proceedings of the 2nd International Conference on Learning Analytics and Knowledge, 252-254, 
http://dl.acm.org/citation.cfm?id=2330661 

31 Siemens, G. (2011). “1st International conference on learning analytics and knowledge 2011: Connecting 
the technical, pedagogical, and social dimensions of learning analytics. 
https://tekri.athabascau.ca/analytics/about 

32 In a distinction reminiscent of Monty Python’s Life of Brian: “BRIAN: Are you the Judean People’s Front? 
REG: F**k off! BRIAN: What? REG: Judean People’s Front. We’re the People’s Front of Judea! Judean 
People’s Front. Cawk.” http://montypython.50webs.com/scripts/Life_of_Brian/8.htm 

33 du Boulay, et al., “What does the research say about how artificial intelligence and big data can close the 


achievement gap?” 270. 
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The Open University’s OU Analyse’ tool, which draws on data from 
across the university (such as student access of online learning materials, 
submission of assessments, and outcomes) to identify students who 
might be at risk of dropping out from their studies—to enable tutors and 
student-support staff to provide appropriate pro-active remedial support. 
In fact, with the fields continually informing and cross-fertilizing each 
other, the distinctions between learning analytics, educational data 
mining, and AIED as a learning sciences research tool are becoming 
increasingly blurry. Often, it simply comes down to the communities who 
are involved in the research and the terminology that they use. Here, as 
we are writing about AIED, we will continue to use AIED terminology. 
One prominent example of ATED as a learning sciences research tool 
has recently been published by the Medical Research Council Cognition 
and Brain Sciences Unit at the University of Cambridge.” The traditional 
grouping of students with learning difficulties in broad categories such as 
ADHD, dyslexia, and autism has long been known to be insufficiently 
helpful, when educators try to improve learning outcomes for individuals. 
For this reason, the Cambridge researchers are investigating the use of 
machine learning to categorize struggling students at a more granular 
level (based on measures of listening skills, spatial reasoning, problem 
solving, vocabulary, and memory). By analyzing data from more than 500 
children, the machine learning revealed four clusters of learning 
difficulties (which had not previously been so clearly delineated): 
difficulties with working memory skills, difficulties with processing 
sounds in words, broad cognitive difficulties in many areas, and typical 
cognitive test results for the student’s age. The researchers found that 
diagnosing struggling learners in terms of these four clusters was both 


** See Herodotou, C., et al. (2017). “Predictive modelling for addressing students’ attrition in higher 
education: The case of OU analyse.” http://oro.open.ac.uk/49470/ and https://analyse.kmi.open.ac.uk 

95 See Astle, D.E., Bathelt, J. and Holmes, J. (2018). Remapping the cognitive and neural profiles of children 
who struggle at school.” Developmental Science. https://doi.org/10.1111/desc.12747 and, for a short summaty, 
https://www.opencolleges.edu.au/informed /learning-strategies /artificial-intelligence-identifies-students- 


struggle-school 
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more accurate and more useful, helping educators address individual 
learning difficulties, than the traditional diagnostic labels. 

We will conclude our brief discussion of AIED as a learning sciences 
research tool with one final example, one that is in the early stages but 
has important potential: the use of machine learning to improve learning 
design. Learning design refers to a range of methodologies “for enabling 
teachers/designers to make more informed decisions in how they go 
about designing learning activities and interventions.” These methods 
are intended to inform decisions about pedagogy (teaching and learning) 
and about ways to support student learning experiences, and can also be 
used to provide core data for learning analytics or educational data 
mining. Most approaches in use in universities’ draw on teachers’ 
professional knowledge of teaching and learning (knowledge that is often 
tacit and thus has had to be elicited from them, which is a non-trivial task 
and can lead to fuzziness and inconsistencies). Instead, the approach 
currently being researched at the Open University involves machine 
learning from thousands of existing module activities to identify 
categories of activities at a highly granular level. Once these learning 
design activity categories are identified, and have been robustly 
authenticated, it should then be possible to correlate the actual learning 
designs of course modules with student outcomes, to help us better 
understand how students learn. In turn, this might inform teachers and 
learning designers about which learning designs (depending on, for 
example, domain, specific subject, duration and level of study) ate most 
effective in practice. 


°° Conole, G. (2012). Designing for Learning in an Open World (v. 4). Springer Science & Business Media. 
https://books.google.co.uk/books?hl=en&lr=&id=gjHNlbc1 BMY C&oi= fnd&pg=PR5&dq=Designing+for 
tlearning+in+an+open+world+ &ots=SwmKc5sSR3&sig=9RUsY FxOKFtZkfxj85WsLkJGcKe 

97 E.g., Cross, S., et al. (2012). “OULDI-JISC project evaluation report: The impact of new curriulum design 
tools and approaches on institutional process and design cultures.” http://oro.open.ac.uk/34140/; Laurillard, 
D., et al. (2013). “A constructionist learning environment for teachers to model learning designs.” Journal of 
Computer Assisted Learning 29 (1): 15-30; Dalziel, J. (ed.), Learning Design. Routledge. 
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AI in Education—A Tentative Summary 


In the previous sections we have discussed a wide variety of existing and 
potential AIED technologies. One way to access this variety is to 
consider the technologies in terms of whether they are mainly student 
teaching (they take a mainly instructionist approach), or student 
supporting (they take a mainly constructivist approach), or teacher 
supporting (they mainly help teachers do what they already do but more 
quickly or with less effort). A summary representation of this is shown in 
the following table. A cursory examination of this table will reveal that 
the categorization provides only a high-level overview, while many of the 
AIED approaches overlap, and most of the technologies could easily 
appeat in another place in the table. It is also likely that over time 
different AIED technologies will merge into multi-capable systems, 
perhaps incorporating sequenced (ITS), Socratic (DBTS), and self- 
directed (ELE) learning in one technology.” 


38 Early examples of this include Holmes, W. (2013). “Level up! A design-based investigation of a prototype 
digital game for children who are low-attaining in mathematics.” (Unpublished PhD thesis, University of 
Oxford) and Rummel, N, et al. (2016). “Transforming learning, empowering learners.” The International 
Conference of the Learning Sciences 1. 
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ITS ELEs ITS+ 
DBTS Automatic writing Automatic writing 
Language evaluation evaluation 
learning apps (formative) (summative) 
Learning network Student forum 
AIED orchestrators monitoring 
ae Language learning Al Teaching 
Applications apps Assistants 
Al Collaborative Al as a research 
learning tool to further the 
Al Continuous learning sciences 
assessment 
Al Learning 
companions 
AIED e Chatbots 
Technologies e AR and VR 
and e Natural Language Processing 
e Adaptivity 


Approaches 


Student teaching, student supporting, and teacher supporting AIED. 


This summary is given more flesh in the following table, 
Characteristics of AIED Technologies. 


——E———E: 
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Type of 
AIED 


Intelligent 
Tutoring 
Systems 


Dialogue- 
based 
Tutoring 
Systems 


Exploratory 
Learning 
Environments 


Automatic 
Feedback and 
Scoring of 
Essays 


ITS+ 


Characteristics 


Step-by-step sequence of 
instruction and tasks. 
Individualized pathways. 
System-determined 
content and pathways. 
Students working with 
computers (or mobile 
devices). 

Individualized feedback. 
Real-time adaptivity. 
Step-by-step dialogue- 
based instruction and 
tasks. 

Individualized 
conversations. 
System-determined 
content and pathways. 
Students working with 
computers (or mobile 
devices). 

Individualized feedback. 
Real-time adaptivity. 
Exploratory tasks. 
Individualized pathways. 
System-determined 
content and pathways, 
with student choice within 
tasks. 

Students working with 
computers (or mobile 
devices). 

Individualized feedback. 
Real-time adaptivity. 
Essays (and other 
assignments) uploaded 
and analyzed by the 
system. 

Some provide 
individualized formative 
feedback (to help 
students improve their 
writing), some only 
summative assessment 
(to score/grade the 
essay). 

Depends on the ITS+. 
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Determined 
by 


System 


System 


System and 
learner 


System 


Target 


For students 


For students 


For students 


For students 
(formative) 


For teachers 
(summative 


Language 
Learning 
Apps 


Chatbots 


Augmented 
and Virtual 
Reality 
Learning 
Network 
Orchestrators 


Collaborative 
Learning 


Student 
Forum 
Monitoring 


Continuous 
Assessment 
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Whole-school 
wraparound ITS. 

Student data visible to 
teacher; superimposed 
above each student via 
augmented-reality 
glasses. 

Back-end ITS functionality 
(AIED as a service) for 
other providers of EdTech 
products. 

Step-by-step sequence of 
instruction and tasks. 
System-determined 
content and pathways. 
Students working with 
computers (or mobile 
devices). 

Individualized feedback. 
Mostly providing 
information. 


Mostly providing access 
to otherwise unavailable 
environments. 

Mostly providing access 
to learning opportunities. 


Facilitating the 
organization of 
collaborative learning. 
Facilitating collaborative 
learning. 

Providing automatic 
feedback to forum posts, 
perhaps making 
connections between 
posts and sentiment 
analysis. 

Assessing student 
competencies on an 
ongoing basis (e.g., 
during talk), rather than 
using tests or exams. 


n/a 


System 


Student (i.e., 
responds to 
student 
questions) 


Mixed 
Mixed (i.e., 
sometimes 
responds to 
student 


requests) 


System 


n/a 


System 


For students 
and teachers 


For students 


For students 


For students 


For students 


For students 


For students, 
and for 
teachers 


For students 
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Al Learning e Potentially, lifelong Student and For students 


Companions learning companions for system 

students. 
Al Teaching e Potentially, Al assistants Teacher and For teachers 
Assistants for teachers. system 


Finally, we might compare all of the AIED technologies with the 
SAMR model discussed in the context section of this book. This 
highlights how most of the near and medium-term advantages of AIED 
are in the augmentation and modification of present-day activities, while 
the long term might see a substantial Holy Grail benefit in redefinition. 


EdTech at Large 
(using the SAMR AIED in Particular 
model) 
Technology allows for e Al removing the need for stop-and- 
eer the creation of new test examinations (i.e. by providing 
Recetiniion tasks, previously continuous highly adaptive 
inconceivable. assessments). 
e AR and VR learning experiences 
Technology allows for e Al Learning Companions 
Modification significant task e Al Teaching Assistants 
redesign. e Alas a Learning Sciences research 
tool. 
e ITS 
e DBTS 
e Exploratory Learning Environments 
Tecnnoleay als wie Automatic Writing Evaluation 
; direct tool substitute, 
pugimem ation with functional ae p 
$ e Language Learning 
improvement. e Chatbots 
e Collaborative Learning support 
e Student forum monitoring 


Substitution 


Technology acts as a 
direct tool substitute, 
no functional change. 


Not applicable (as of this writing) 


AIED and the SAMR model. 
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The Social Consequences of AI in Education 


As we have seen, the application of AI in educational contexts is growing 
rapidly. In this book, we have explored the various AI techniques being 
used, the applications that have been in development for almost fifty 
years, and the futuristic possibilities that are becoming ever more likely 
(whatever our personal values). 

Clearly, AIED has achieved some notable successes, while the 
conceivable applications are at the least intriguing. However, AIED’s 
potential impact on students, teachers and wider society is yet to be fully 
worked out. This is true of issues as broad as accuracy, choice, 
predictions, privacy, teachers’ jobs, and what we should be teaching 
school and university students.” But it is especially true for AIED’s 
emerging ethical questions: “Around the world, virtually no research has 
been undertaken, no guidelines have been provided, no policies have 
been developed, and no regulations have been enacted to address the 
specific ethical issues raised by the use of artificial intelligence in 
education.” 

In any case, one wonders why, if AIED is so effective, has it not yet 
been widely adopted by schools, universities and training companies? In 
fact, it is not yet even clear whether the AI technologies being imported 
into education are actually up to the task. For many years, non-AI 
technologies in educational settings have been critiqued. The question is 
whether ATED is destined to become the latest computer technology to 


. 401, 402 . 
be oversold yet underused in classrooms.®™ ° We also need to considet 


™ E.g., “Machine learning: universities ready students for AI revolution,” https:// www-timeshighereducation- 
com libezproxy.open.ac.uk/news/broader-four-year-degrees-offered-in-response-to-ai-tevolution and “The 
most important skills for the 4th industrial revolution? Try ethics and philosophy.” 
https://www.edsurge.com/news/2018-1 0-06-the-most-important-skills-for-the-4th-industrial-revolution-try- 
ethics-and-philosophy 

* Holmes, W., et al. (2018). “Ethics in AIED: Who cares?”In Artificial Intelligence in Education (ed. Rosé, C.P., 
et al.). 19th International Conference Proceedings, Part II. https://doi.org/10.1007/978-3-319-93846-2 

#1 Cuban, L. (2001). Oversold and Underused: Computers in the Classroom. Harvard University Press. 

“2 “Pretty much all edtech sucks. And machine learning is not going to improve edtech.”—Al Essa, McGraw- 
Hill Education; and “I don’t see a child sitting in front of an Alexa and being taught, because there is a whole 


other set of cues they need to learn. I don’t see machine learning reaching that point.”—Janel Grant. Both 
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what might happen, what might be the impact on individual learners, if 
ineffective AI techniques (or biased data sets) are used in classrooms (for 
example, what might happen if the face recognition technology that 
achieved 95% false positives for the UK’s Metropolitan Policet? was 
used in classroom monitoring)? Meanwhile, there are few examples of 
cumulative or replicable AIED research: the field is developing so rapidly 
while AIED data sets and algorithms tend to be (jealously?) guarded. 
There is also little available robust evidence of the efficacy at scale of the 
rapidly increasing numbers of AIED tools. Even those, such as Mathia 
and Assistments, that do have some evidence, have typically been 
compared with business as usual rather than with another technology 


4 


that has at least some level of comparability.“ The purported 


effectiveness of many other tools may be due to their novelty in 
5 


classrooms," rather than anything to do with the AI employed—we 


simply do not have the evidence to say one way or another. 


The Implications of AIED Technologies for Classrooms 


We began our AIED journey with intelligent tutoring systems, which as 
we saw ate the most common of AIED applications, and which we will 
now use to scaffold and highlight some social consequences of AI 
applied to education that deserve more detailed attention. It has long 
been recognized that AI by design amplifies hidden features of its initial 
data and effectively reinforces its underlying assumptions. In particular, if 
the algorithms “are trained on data which contains human bias then of 
course the algorithms will learn it, but furthermore they are likely to 


amplify it. This is a huge problem, especially if people assume that 


quoted in Johnson, S. (2018). “What can machine learning really predict in education?” EdSurge. 
https://www.edsurge.com/news/201 8-09-26-what-can-machine-learning-really-predict-in-education 

403 The Independent, May 2018. https://ind.pn/2InMfGf 

+ Holmes, W., et al. Technology-Enhanced Personalised Learning, 65 and 68. 

“5 Schomaker, J. and Meeter, M. (2015). “Short- and long-lasting consequences of novelty, deviance and 
surprise on brain and cognition.” Neuroscience €> Biobehavioral Reviews. 
https://doi.org/10.1016/j.neubiorev.2015.05.002 
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algorithms are impartial.”*® In this respect, both rule-based and machine 
learning ITSs are no different. Their very design, their implementation of 
step-by-step instructionist methods focused on a knowledge curriculum 
while ignoring contextual and social factors, amplifies existing yet 
contested assumptions about effective approaches to teaching, and even 
to what it means to learn.*” 

ITSs also embody a usually unacknowledged paradox, the dependence 
of personalized approaches to learning on identifying what is collective 


or average. 


[ITS] recommend lessons to users based on how other 
learners on the system have performed. These systems 
“learn” each student by presuming them to be similar to 
others... We herald an intervention as a success if [an 
efficacy study shows that] it works on average, discarding 
the nuances of why it may work for some students more 
than others, and to what degree. [In summary], the 
individual struggle of the individual learner is easily lost in 
the noise.408 

In other words, focusing on the average to determine an appropriate 
intervention is inevitably limiting: if a robust study shows that one 
approach is mote effective on average compared with a second approach, 
the second approach is likely to be fully rejected, despite the fact that it 
might be more effective for particular individuals or groups. 

ITSs by design also can reduce student agency. Although constrained 
by the curriculum (as decided by local or national policy-makers), it is 
generally the ITS (its algorithms and student models) and, at a higher 
level, the ITS designers, that determine what should be learned, in what 


order and how; while the student is given little choice but to follow the 


“6 Douglas, L. (2017). “AT is not just learning our biases; it is amplifying them.” Medium. 
https://medium.com/@laurahelendouglas/ ai-is-not-just-learning-our-biases-it-is-amplifying-them- 
4d0dee75931d 

“7 Instructionism “is based on cognitive learning theories that center on teaching as education performed by a 
teacher. In the view of instructionism, instruction has to be improved in order to achieve better learning 
results.” Seel, N.M., ed. (2012) Encyclopedia of the Sciences of Learning. Springer. 

*$ Mubeen, J. (2018). “When ‘personalised learning’ forgets to be ‘personalised.”” Medium. 
https://medium.com/@fjmubeen/when-personalised-learning-forgets-to-be-personalised-48c3558e7425 
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TTS-determined individual pathway (it also in some sense makes the 
teacher somewhat redundant—it is the system, not the teacher, that 
decides what is best for a student to learn). For example, most ITS begin 
with the basics, before guiding the individual student through tasks that 
take them step-by-step towards mastery targets, minimizing failure along 
the way. However intuitively appealing, the assumptions embodied in this 
instructionist approach” also ignore the value of other approaches 
researched in the learning sciences (such as collaborative learning, guided 
discovery learning, blended learning, and productive failure).*"° 

ITSs also raise issues centered on the selection of data, raising 
complex issues centered on trust.“!! For example, it has been argued that 
there is no such thing as raw data:*" data used in any analysis has been 
pre-selected (it is not possible to include all data generated by a system in 
its computations), and these choices are inevitably subject to conscious 
or unconscious, explicit or implicit, selection biases,‘ Similarly, the 
algorithms chosen or developed raise additional issues, such as those 
centered on the accuracy and implications of their predictions (if the 
computation is incorrect, are students being guided away from their best 
interests, and how do we ensure that mistakes ert on the side of failing in 
the least harmful way?), the increasing focus on inferring and responding 
to the students’ affective states (are a student’s innermost feelings not 


414 


private anymore?)*'* and the usual focus on teaching the type of 


knowledge that is the easiest to automate and thus potentially the least 


useful in the long-term for students.*!> 


————— 


“© Gagné, Conditions of Learning and Theory of Instruction. 

+ Dean Jr., D. and Kuhn D. (2007). “Direct instruction vs. discovery: The long view.” Science Education 91. 
https://doi.org/10.1002/sce.20194 

a Bigs, https://www.theatlantic.com/magazine/archive/201 8/11 /alexa-how-will-you-change-us/570844/ 
2 Gitelman, L., et al. (2013). “Raw Data” Is an Oxymoron. MIT Press. 

“3 “Data is easily gotten, but it has a lot of bias in it.” John Behrens (Pearson), quoted in Johnson, What Can 
Machine Learning Really Predict in Education? https://www.edsurge.com/news/2018-09-26-what-can-machine- 
learning-really-predict-in-education 

#14 “Tech firms want to detect your emotions and expressions, but people don’t like it.” 
hups://theconversation.com/tech-firms-want-to-detect-your-emotions-and-expressions-but-people-dont- 
like-it-80153 

45 Rose Luckin quoted in https://www.jisc.ac.uk/news/the-ai-revolution-is-here- 17-aug-2018 
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In any case, as we discussed eatlier, the efficacy of ITSs in teal 
educational settings remains to be confirmed (although many have been 
shown to be broadly effective when compared against usual classroom 
teaching).*"° Indeed, one ITS, Summit Learning," which was developed 
by engineers from Facebook and is being used in around 400 schools, 


has been the focus of student protests and boycotts. 


Unfortunately we didn’t have a good experience using the 
Program, which requires hours of classroom time sitting in 
front of computers... The assignments are boring, and it’s 
too easy to pass and even cheat on the assessments. 
Students feel as if they are not learning anything and that 
the program isn’t Preparing them for the Regents exams 
they need to pass to graduate. Most importantly, the entire 
Program eliminates much of the human interaction, 
teacher support, and discussion and debate with our peers 
that we need in order to improve our critical thinking. 
Unlike the claims made in your promotional materials, we 
students find that we are learning very little to nothing. It’s 
severely damaged our education, and that’s why we walked 
Out in protest.418 


Finally, ITSs typically set themselves up as doing at least some of the 
job of teachers, increasingly more effectively than teachers, thus 
questioning the role of teachers in future classrooms.“ As we have seen, 
the ambition of many researchers is to relieve teachers of the burdens of 
teaching (such as monitoring progress and marking assignments), 
enabling them to focus on the human aspects of teaching (such as social 
engagement). In fact, “AI cannot create, conceptualize, or manage 
complex strategic planning; cannot accomplish complex work that 


requires precise hand-eye coordination; cannot deal with unknown and 


#16 du Boulay, B. “Artificial intelligence as an effective classroom assistant.” IEEE Intelligent Systems 31. 
https://doi.org/10.1 109/MIS.2016.93 

#7 https://www.summitlearning.org 

48 The Chan Zuckerberg Initiative funded the Summit Learning project and disputes these claims, 
https://www.washingtonpost.com/education /2018/11/17/ students-protest-zuckerberg-backed-digital- 
learning-program-ask-him-what-gives-you-this-right/ Pnoredirect=on&utm_term=.27d5e322aclc 

* At least one ITS company appeared to pivot from attempting to sell their product into schools, because 
teachers were unsure why they should use a technology that did their job instead of them. 
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unstructured spaces, especially ones that it hasn’t observed; and cannot, 
unlike humans, feel or interact with empathy and compassion... tasks 
that can only be done by a human teacher. As such, there will still be a 
great need for human educators in the future.” But, on the other hand, 
if we (students, educators, and parents) do not critically engage, perhaps 
ATED might lead to fast-food chef, script-driven classroom managers*” 
rather than teachers, while the AI deals with all of the cognitive demands 
of teaching (a dystopian scenario that is only some short steps away from 
removing humans from teaching entirely). 

Naturally, there are many examples of ITSs that challenge at least 
some of these issues (such as Mathia, whose developers recommend it is 
delivered in a blended context). We have also looked at alternative 
approaches, such as DBTSs (that prioritize a Socratic, albeit step-by-step, 
approach rather than an instructionist approach to learning) and AI- 
driven ELEs (that prioritize a guided-discovery approach to learning). 
And we have considered alternative ways in which AI is being or might 
be used in innovative ways, that have the potential to step outside 
dominant educational practices: for example, relatively simple AI that 
enables students to connect to their choice of human tutors (to get 
support on what they want to learn), and complex AI that provides a 
lifetime learning companion dedicated to their needs. Yet even these 
approaches depend on huge amounts of personal data and efficient 
algorithms, raising ptivacy and ethical issues that have yet to be fully 


considered. 


oo ÃĂōŐ— 


420 https://www.linkedin.com/pulse/10-jobs-safe-ai-world-kai-fu-lee, Also see, “Intelligent machines will 
replace teachers within 10 years, leading public school head teacher predicts.” 
https://www.independent.co.uk/news/ education /education-news /intelligent-machines-replace-teachers- 
classtoom-10-years-ai-robots-sir-anthony-sheldon-wellington-a7939931.html; “Could artificial intelligence 
replace our teachers?” hetps://www.educationworld.com/could-attificial-intelligence-replace-our-teachers; 
and “Why artificial intelligence will never replace teachers,” https://www.thetechedvocate.org/artificial- 
intelligence-will-never-replace-teachers 

#21 As we mentioned earlier, one ITS developer has suggested that the sophistication of their AIED means that 
teachers only need to play an auxiliary role, working like fast-food chefs (“KFC-like”) to strictly regulated 
scripts. 
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The Ethics of AIED 


Indeed, the ethics of AI applied in education, although left to last in this 
book, requires urgent attention. For example, one school 


has installed facial recognition technology to monitor how 
attentive students are in class. Every movement of pupils 

is watched by three cameras positioned above the 
blackboard... Some students are already changing their 
behaviour due to the increased monitoring.... “I don't dare 
be distracted after the cameras were installed in the 
classrooms. It's like a pair of mystery eyes are constantly 
watching me." The system works by identifying different 
facial expressions from the students, and that information 
is then fed into a computer which assesses if they are 
enjoying lessons or if their minds are wandering... The 
computer will pick up seven different emotions, including 
neutral, happy, sad, disappointed, angry, scared and 
surprised. If it concludes that the student is distracted with 
other thoughts during the class, it will send a notification 
to the teacher to take action.’422 


This example of AI being used to maximize student attention is from 
China. However, before we dismiss it as a culturally-specific 
phenomenon, we should remember that ALT Schools’ also uses AT- 
driven classroom cameras to monitor student behavior (while in the UK, 
“tens of thousands of pupils aged as young as five are at risk of being 
spied on through their webcams..., often without students or their 
parents ever knowing” ).*” This is not to say that the use of AI to analyze 
classroom video feeds is by definition unethical. For example, researchers 
at the University of Pittsburgh are using AI and classroom videos to help 
better understand how the quality of classroom talk, the liveliness of 


— 


#2 https://www.telegraph.co.uk/news/2018/05/17/) chinese-school-uses-facial-recognition-monitor-student- 
attention/ 

23 https://https://www.telegraph.co.uk/technology/201 8/12/15/children-young-5-risk-spied-webcams- 
using-school-software 
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discussion, and the level of student engagement contributes to effective 
learning, to inform better approaches to teaching.** 

On the other hand, there are examples of AI companies*” collecting 
huge amounts of student interaction data, in order to use machine- 
learning techniques to “search for patterns.” The aim, naturally, is to 
“improve student learning by teaching the software to pinpoint when 
children are feeling happy, bored, or engaged.”*® Nonetheless, this 
approach is controversial, with such data collection being characterized 
as “borderline mental-health assessments..., [that] encourage a view of 
children as potential patients in need of treatment.”*”” 

The reality is that, while the range of AI techniques and technologies 
researched in classrooms and discussed at conferences ate extensive and 
growing, the ethical consequences are rarely fully considered (at least, 
there is very little published work considering the ethics). In fact, most 
AIED research, development, and deployment has taken place in what is 
essentially a moral vacuum (for example, what happens if a child is 
unknowingly subjected to a biased set of algorithms that impact 
negatively and incorrectly on their school progress?). In particular, AIED 
researchers are working without any fully worked out moral groundings. 

In fact, as we have seen, AIED techniques raise an indeterminate 
number of self-evident but as yet unanswered ethical questions. To begin 
with, as with mainstream AI, concerns exist about the large volumes of 
data collected to support AIED—albeit data that is collected with the 


best of intentions (such as the recording of student competencies, 


#4 Kelly, S., Olney, A.M., Donnelly, P., Nystrand, M., and D’Mello. S.K. (2018). “Automatically measuring 
question authenticity in real-world classrooms.” Educational Researcher 47. 
https://doi.org/10.3102/0013189X18785613 

435 E.g., https://www.algebranation.com 

#% “How (and why) ed-tech companies are tracking students' feelings.” 
https://mobile.edweek.org/c.jsp?cid=25919761 &bcid=25919761 &rssid=25919751 &item=http%3A%2F%2 
Fapi.edweek.org%2Fv1%2Few%2Findex.html%3Fuuid=C08929D8-6E6F-11E8-BE8B-7F0EB4743667 

#7 Jane Robbins, American Principles Project Foundation, quoted in preceding note, “How (and why) ed- 


tech companies are tracking students’ feelings.” 
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emotions, strategies, misconceptions, and screen usage," to better help 
students learn). Who owns and who is able to access this data, what are 
the privacy concerns, how should the data be analyzed, interpreted, and 
shared, and who should be considered responsible if something goes 
wrong? In a parallel domain, healthcare, the use of personal data can be 
contentious and is frequently challenged*” —but this has yet to happen 
noticeably in education. 

However, while data raises major ethical concerns for the field of 
AIED, AIED ethics cannot be reduced to questions about data. Other 
major ethical concerns include the potential for bias*” (conscious or 
unconscious) incorporated into AI algorithms (i.e. how the data is 
analyzed)*' and into AIED models (what aspects of a domain are 
assumed worth learning, what approaches to pedagogy are assumed to be 
most effective, and what student information is assumed to be the most 
pertinent?). On the other hand, if a computers decisions are 
indistinguishable from that of a human, or at least from a panel of human 
experts (because humans ate well known to sometimes disagree, for 


example when marking essays),*” perhaps those decisions should be 


428 “FaceMetrics lands $2 million to gamify kids’ screen time and track immersion with AI.” 
https://venturebeat.com/2018/06/13/facemetrics-lands-2-million-to-gamify-kids-screen-time-and-track- 
immersion-with-ai 

42 For example, https://www.bbc.co.uk/news/technology-46206677: “A controversial health app developed 
by artificial intelligence firm DeepMind will be taken over by Google ...” Lawyer and privacy expert Julia 
Powles [said]: “DeepMind repeatedly, unconditionally promised to ‘never connect people's intimate, 
identifiable health data to Google.’ Now it's announced... exactly that. This isn't transparency, it's trust 
demolition.” 

#0 “[A]s algorithms play an increasingly widespread role in society, automating—or at least influencing — 
decisions that impact whether someone gets a job or how someone perceives her identity, some researchers 
and product developers are raising alarms that data-powered products are not nearly as neutral as scientific 
thetoric leads us to believe.” Kathryn Hume, integrate.ai, quoted in “AI needs debate about potential bias,” by 
Carole Piovesan, https://www.lawtimesnews.com/article/ai-needs-debate-about-potential-bias-15180. Also 
see, The Fairness Toolkit, https://unbias.wp.horizon.ac.uk/fairness-toolkit 

1 A recent survey by The Pew Research Center found that “the public is frequently skeptical of [algorithms] 
when used in various real-life situations. ... [with] 58% of Americans feel[ing] that computer programs will 
always reflect some level of human bias.” http://www.pewinternet.org/2018/11/16/public-attitudes-toward- 
computer-algorithms/ 

#2 To give an anecdotal example, a Masters thesis written by one of the authors at a prestigious university was 


marked as a distinction by one professor and a fail by another. 
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accepted. *” 


Nonetheless, each decision that goes into constructing these 
algorithms and models might impact negatively on the human rights of 
individual students (in terms of gender, age, race, socio-economic status, 
income inequality, and so on)—at present we just do not know whether 
or not they will. 

But these particular AI ethical concerns, centered on data and bias, are 
the “known unknowns,” and are the subject of much research and 
discussion in mainstream AI tesearch.*** What about the “unknown 
unknowns,” those ethical issues raised by the application of AI in 
education that have yet to be even identified? 

AIED ethical questions include (there are many more): 


° What are the criteria for ethically acceptable AIED? 

e How does the transient nature of student goals, interests and 
emotions impact on the ethics of AIED? 

e What are the AIED ethical obligations of private 
organizations (developers of AIED products) and public 
authorities (schools and universities involved in AIED 
research)? 

e How might schools, students, and teachers opt out from, or 
challenge, how they are represented in large datasets? 


* From another perspective, the UCLA law professor Eugene Volokh argues that “a computer should be 
accepted if a panel of humans thinks the opinions it writes are on par with or better than those written by a 
human judge...” (https://www.axios.com /artificial-intelligence-judges-Oca9d45£-£7d3-43cd-bf03- 
8bf2486cf£36.html) 

4 E.g., Ada Lovelace Institute (https://www.adalovelaceinstitute.org), AI Ethics Initiative 
(https://aiethicsinitiative.org), AI Ethics Lab (http://www.aiethicslab.com), AI Now 
(https://ainowinstitute.org), DeepMind Ethics and Society (https://deepmind.com/applied/deepmind- 
ethics-society), and the Oxford Internet Institute (https://www.oii.ox.ac.uk/blog/can-we-teach-morality-to- 
machines-three-perspectives-on-ethics-for-artificial-intelligence). Also see Winfield, Alan F. T., and Jirotka, M. 
(2018). “Ethical governance is essential to building trust in robotics and artificial intelligence systems.” Phil, 
Trans. R. Soc. A 376. https://doi.org/10.1098/rsta.2018.0085 And see “Top 9 ethical issues in artificial 
intelligence.” https://www.weforum.org/agenda/201 6/10/top-1 0-ethical-issues-in-artificial-intelligence 
“Establishing an AI code of ethics will be harder than people think.” 
https://www.technologyreview.com/s/6 1231 8/establishing-an-ai-code-of-ethics-will-be-harder-than-people- 
think, and Willson, M. (2018). “Raising the ideal child? Algorithms, quantification and prediction.” Media, 
Culture © Society, 5. https://doi.org/10.1177/0163443718798901 
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e What are the ethical implications of not being able to easily 
interrogate how AIED deep decisions (using multi-level neural 
networks) are made? 


Strategies are also needed for ameliorating risk, since AI algorithms 
are vulnerable to hacking and manipulation (as the Facebook—Cambridge 
Analytica data scandal showed was mote than possible): “It’s impossible 
to have personal privacy and control at scale, so it is critical that the uses 
to which data will be put are ethical — and that the ethical guidelines are 
clearly understood.” Where AIED interventions target behavioral 
change (such as by nudging individuals towards a particular behavior or 
course of action), the entire sequence of AIED-enhanced pedagogical 
activity also needs to be ethically warranted. Finally, it is important to 
recognize another perspective on AIED ethical questions: in each 
instance, the ethical cost of inaction and failure to innovate must be 
balanced against the potential for AIED innovation to result in teal 
benefits for learners, educators, and educational institutions. 

In short, the ethics of AIED is complicated. 

As is likely already clear, the authors of this book are excited by what 
AT has to offer teaching and learning ... but we are also very cautious. We 
have seen an extraordinary range of AIED approaches (from Mathia, 
AutoTutor and Betty’s Brain, to the Ada chatbot, OpenEssayist, and 
Lumilo, and more) and some amazing future AIED possibilities (from 
the end of exams, to lifelong learning companions, and AI teaching 
assistants). However, we have also identified a range of critical issues that 
need to be addressed before AI becomes an acceptable integral part of 
everyday learning. 

Most important, the ethics of AIED need to be fully worked out—a 
non-trivial task that requires the involvement of a wide range of 
stakeholders (ftom students to philosophers, teachers to policymakers, 


and parents to developers). We (teachers, policymakers, and learning 


+5 Tarran, B. (2018). “What can we learn from the Facebook—Cambridge Analytica scandal?” $. wgnificance 15 (3): 
4-5. 
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scientists) need to understand the key issues raised by the collection of 
data (such as the choice of what data to collect and what data to ignore, 
the ownership of data, and data privacy). We also need to understand the 
computational approaches being applied (what decisions are being made, 
what biases are creeping in, and how do we ensure that decisions are 
‘correct and transparent?).**° This much is self-evident, which is why so 
many initiatives to both determine and govern the ethics of AI have been 
established around the world. 

However, we also need to have a thorough understanding of the 
ethics of education, of teaching and learning (the ethics of particular 
approaches, curriculum choices, focusing on averages, the allocation of 
available funds, and much more besides), another non-trivial task. For 
without that, how will we know what might happen when these three 
areas (data, computation, and education) collide? 

This returns us to our introduction, and is hopefully our main 
takeaway. Whether we welcome it or not, AI is increasingly being used 
widely across education and learning contexts. We can either leave it to 
others—the computer scientists, AI engineers*” and big tech 
companies—to decide how artificial intelligence in education unfolds, or 
we can engage in productive dialogue. It is up to each of us to decide 
whether we acquiesce, take what we are given, or whether we adopt a 
critical stance, to help ensure that the introduction of AI into education 


reaches its potential and has positive outcomes for all. 


436 See, Miller, T. (2019). “Explanation in artificial intelligence: Insights from the social sciences.” Artificial 
Intelligence 267. https://doi.org/10.1016/j.artint.2018.07.007 
487 “You and AI-,achine learning, bias and implications for inequality.” https://royalsociety.org/science- 


events-and-lectures/2018/07/you-and-ai-equality 
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Appendix 1 


Connections Between Topics and Concepts 

When designing a curriculum, it is important to make sure all content 
serves to highlight the (Threshold) Concepts of the relevant knowledge 
areas, across levels. Table X shows the relationship between sample 
Chemistry Topics and relevant Chemistry Core Ideas. 


Sample Topics (Chapter Headings) Relationship to Core ideas 
Periodic Trends Atomic/Molecular Structure and Properties: Trends that repeat across rows and down columns arse from the atomic 

structure, 

Electrostatic and Bonding Interactions: Electrostatic interactions {that give rise to effective noclear charge) between 
subatomic partidos explain most periodic trends, 

Energy: Quantum: Patterns in ionization energies arise from the quantized nature of the energy levels. Shiekling by core 
electrons determines the effective nuclear charge experienced by valence electrons and results from the relationship 
between the quantized energy levels in atoms and the associated shell structure. 


Change and Stability in Chemical Systems: Balance between attractive and repulsive forces determines the size of atoms, 

Solutions Atomic/Molecular Structure and Properties: Solubility of a substance depends upon the molecular level structure of both 
solute and solvest. 

Electrostatic and Bonding Interactions: Interactions between solvent and solute partly determine the solubility. 

Energy: Macroscopic: Temperature changes that take place when a substance dissolves depend on the energy required to 
overcome interactions and energy released when new interactions are formed. 

Change and Stability in Chemical Systems: Solubility of a substance depends an the total entropy change when a 
substance dissaives. 

Phases and Phase Changes Atamic/Molecular Structure and Properties: Melting/boiling point of a substance depends on its moleculartevel 
structure. 

Electrostatic and Bonding Interactions: Types and strengths of interactions at the molecular level influence melting? 
boiling points. 

Energy: Macroscopic: Energy changes associated with phase changes are determined by the types and strengths of 
interactions. 

Change and Stability in Chemical Systems: Phase change temperatures depend upon the energy transfer between system 
and surroundings and the corresponding entropy changes. 

Kinetics Alomic/Molecalar Structure and Properties: Rates of chemical reactions depend on the structures of the reacting species 
aad the protubility of the mactants beag properly oriented in a collision, The mechanism for the reaction also depends on 
the molecular structare. 

Electrostatic and Bonding Interactions: Rates of chemical reactions depend on the strengths of the interactions between 
the reacting species and the strengths of the bonds within them. 

Energy: Molecular and Macroscopic: Rates of chemical reactions depend on the activation energy (which in turn depends 
ou the stracie and interactions) and an the kinetic energy of the colliding reactant molecoles. 

Change and Stability in Chemical Systems: Kinetics is the study of how and why chemical change occurs. Competing rates 
of forward and reverse reactions control the extent of a reaction and when balacced equilibrium is reached. 

Thermochemistry Atomic/Molecular Structure and Properties: Some bonds and interactions are broken and new ones formed in a chemical 
reaction, 

Electrostatic and Bonding Interactions: Types and strengths ef bonds and interactions depend on the structure and 
polity of the molecules involved, ‘The strengths of the interactions can be predicted from electrostatic considerations. 


č in a chemical reaction is a balance between the energy required ta 


Energy: Molecular and Macroscopic: Energy 
d when new bonds and interactions are formed. 


break bonds and interactions, and the energy rel 
Change and Stability in Chemical Systems: Whether a chemical reaction occurs or not depends on the total entropy 
change, which can he determined frem a consideration ef enthalpy and entropy changes of the system. 


Topics and their alignment to concepts. 
Source: Cooper, Posey, and Underwood 


In the table above, it may seem that some of the Core Ideas are 


themselves potential Subjects, in that they are good ways of organizing 
content. This is a consequence of examining knowledge at such a fine- 
gtained scale; Topics can be a mix of Concepts and Content. However, 
this provides an important criterion for extracting from the curriculum 


elements that are dense in content but not in concept. 
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For example, one can imagine the Krebs cycle as a Topic in Cellular & 
Molecular Biology, but, although it clearly is relevant to other concepts, it 
is dse/f not a useful concept. Contrasting that with the Topic of 
Succession in Ecology, it becomes evident that although Succession may 
be a chapter title in a textbook, it is also itself an important concept of a 
type of dynamic that happens in ecosystems over time (likely a Threshold 
Concept of Ecology). Perhaps, then, the Krebs cycle should not be 
memorized for its own sake, and instead only be studied to the degree 
that it serves other concepts, whereas Succession may be a fruitful stand- 


alone Topic of Ecology. 


Evolution of Content 


8 12 18 27 


ARTIUIMIETIC 


Trivium Quadrivium 
The classical Greek education frameworks 

The most influential of early Western formulations of the curriculum 
were the Trivium and Quadtivium—a medieval revival of classical Greek 
education theories that defined the seven liberal atts fot university 
education: grammar, logic, rhetoric, astronomy, geometry, arithmetic, and 
music. 

Changes have occurred very slowly, with some subjects becoming 
optional (astronomy, ethics, Latin, etc.), or taught at tertiary education 
level (rhetoric, oratory as “communications”). Knowledge discipline 
standards in secondary education in the United States were first 
established in 1893 by the Committee of Ten, led by Charles Eliot, the 
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president of Harvard University and sponsored by the National 
Education Association. He convened ten committees of education 
experts, led mostly by college presidents and deans, and charged them 
with defining the standardized curriculum requirements for all public 
secondary schools. 


‘able ELI of the Report of the Committee of Ten 


Ist SECONDARY SCHOOL YEAR. 


2ND SECONDARY SCHOOL YEAR. 


BAG copii suuceemnceds Ss p TRUD srsiusnesucmrveainesugan 4p 
English Literature; 2p. $ CEERI oirra KAE 5p 
“ Composition, 2 Pt Be English Literature, 2 p, 

German [or French] ........ 5 p. gi Composition, 2 p,f "* ap. 

Algebra: eymar sn oe.» 4p. German, continued ......... 4p. 

History of Italy, Spain, and French, begun 2... ...4...,5 5p. 
PEARCE... oS EAT 3 p- Algebra, * 2 p. 

Applied Geography (European Geometry, 2 p: t SR HANES Ep 
political ~— continental and Botany or Zodlogy. 21.5.2... 4p, 
oceanic flora and fauna) .. 4 p- English History to 1688 ...., 3 p. 

25 p. 33 p. 
* Option: of book-keeping and 
commercial arithmetic. 
3RD SECONDARY SCHOOL YRAR. 4TH SECONDARY SCHOOL YEAR, 


Latin 


` LA caninin 4p. 
Greek . CIOE riina A EEE A A 4 p. 
English Literature, 2 P English Literature, 2 p 
* Composition, 1 p 4p. “Composition, 1 p: 4p 
Rhetoric, lp “ Grammar, 1 p: 
GII mp arescenimers nnd nan 4 p. NONE aR r i 4p 
Brendl aee EO cess, 4p. Freeh issn cmemunarnqere sores, 4p 
Algebra,* 2 oy 4 ‘Trigonometry, 
Geometry, AT AA aaa P. HirherAlesbra, ? SAS BEES 2p 
Physics iaai tinon aain aAa i aiaa 4 p. ENTS aeaio nooti os 4p. 
History, English and History (intensive) and Civil 
PEARS ETA eai RA 3 p. Government ..........., 3 p. 
Astronomy, 3p. 1st i yr. Geology or Physiography, 
Meteorology, 3 p: 2nd 1% yr. } Ppi 4 p. Ist 1% yr. 
34 p. Anatomy, Physiology, and oP 
Hygiene, 4 p. 2nd 14 yr. 


* Option of book-keeping and 3 
| GRennEeeint arithmetic. dB: 


The Report of the Committee of Ten 


To various degrees, these early education discipline standards (minus 
the Greek, Latin, and other specific language requirements) are still 
teflected in secondary school graduation requirements in many education 
systems today. 

Thus the traditional disciplines taught in most education systems 
around the world are: 
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e Languages (domestic) 

e Mathematics (arithmetic, geometry, algebra) 

e Science (biology, chemistry, physics) 

e Languages (foreign) 

e Social studies (history, geography, civics, economics, etc.) 
° Arts (performing & visual) 

° Wellness (often physical education in particular) 


Within the disciplines, the pattern is similar; areas of knowledge are 
added on when they became common enough, but nothing has been 
majorly redesigned. In math for example, the curriculum reflects the 
monumental contribution of Leonardo de Pisa (Fibonacci) who wrote 
the book Liber Abbaci in 1202 to help merchants transition from using 
Roman numerals to using the ten digits of the Hindu-Arabic system, and 


to learn things like ratio and proportion, rate, linear equations, etc. 


Greek, Latin 


Reading, Writing 
“ssn 


Literature 


sm oa e ee | 


Language 


Oratory 


Rhetoric 


Humanities 


Arithmetic 
Astronomy 
pa "Algebra, Trigonometry, Calculus 
Biology, chem istry, Physics 


oe Ree om mm ae 


STEM 


Ancient Early Berauek Modern 
Greece & Christianity & Enlight A Industrial Today 
Rome Middle Ages MENEN Era 


®© 2013 Center for Curriculum Redesign — AN Rights Reserved & Center for Curncutum Redsaign 


The evolution of content over time. Source: CCR 
It has become clear that what was relevant for 13th century merchants 
is not relevant to 21st century students. Progress in human endeavors has 
increased exponentially, and although Trigonometry and Calculus were 
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added to the curriculum, more modern subjects, such as Robotics and 
Entrepreneurship, do not fit into the current over-ctowded system. 
Furthermore, standardized assessments are placing pressure in particular 
on language and mathematics, thereby further narrowing the scope of 
school subjects. In the image below, Social Studies, Science, Recess, 


Art/Music, and PE have by and large been squeezed out to make room 
for ELA and Math in the United States. 


58 ELA 

Percent of districts 
that decreased time 45 Math 
Social Studies 36 Percent of districts 

that increased time 
Science 28 
ArtiMusic 46 
Recess 
PE $ 

400 50 0 50 100 


Percentage of districts in the US with change in allotted time per discipline 
since 2001 Source: Center for Public Education 


Cross-Cutting Themes 


Environmental Literacy 


As discussed earlier, humankind is fast approaching or may have 
sutpassed a number of our planet’s ecological limits, and to avoid future 
majot environmental crises or ecological disasters, every citizen must 
have a basic understanding of the fundamentals of environmental science 
and the impacts of our societies on the long-term sustainability of 
humankind. 

The Partnership for 21st Century Skills defines the components of 
environmental literacy as the abilities to: 


im 
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Demonstrate knowledge and understanding of the environment and 
the circumstances and conditions affecting it, particularly as they relates 
to air, climate, land, food, energy, water and ecosystems. 


° Demonstrate knowledge and understanding of society’s 
impact on the natural world (e.g, population growth, 
population development, resource consumption rate, etc.) 

° Investigate and analyze environmental issues, and make 


accurate conclusions about effective solutions. 


e Take individual and collective action towards addressing 
environmental challenges (e.g., participating in global actions, 
designing solutions that inspite action on environmental 


issues). 
Global Literacy 


Our global community continues to grow more interconnected, and it 
is no longer enough to learn from the perspective of only one country. 
To be educated for the twenty-first century, every student now needs to 
learn each subject from a variety of cultural perspectives from around the 
world. This means for example that world history includes histories from 
countries all over the world, math class discusses relevant Eastern (Arab, 
Indian, and Chinese) mathematicians, not just Western ones, and 
students ate prompted to critically examine their cultural biases and 
perspectives, and develop understanding and acceptance of other 
viewpoints. Throughout the curriculum, students should learn to see 
individual issues within the context of their global socio-cultural 
significance, gain an international awareness, and a deep appreciation of 
cultural diversity. 


Civic Literacy 


Humans and education systems exist within societies, and the main 
ways to interact most directly with society is through law and policy. It is 
important that students feel connected to their society and feel that it’s 
possible to effect change. In the coming years, there are going to be 
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many issues that will need to be discussed publicly and decided at the 
societal scale, and civic literacy will therefore become increasingly 
relevant. To make sure that what is learned is school can be transferred 
to those conversations when necessary, it will be helpful to explicitly 
highlight the ways that law and policy connect with course content, and 
the kinds of questions facing society in the near future that need to be 
considered from an informed perspective. According to the Partnership 
for 21st Century Skills, civic literacy is: 


° Participating effectively in civic life through knowing how to 
stay informed and understanding governmental processes. 

° Exercising the rights and obligations of citizenship at local, 
state, national and global levels. 

° Understanding the local and global implications of civic 
decisions. 


They also recommend several resources to learn more about civic 


literacy 
Information Literacy 


According to Eric Schmidt, CEO of Google, every two days we create 
as much information as was created from the dawn of civilization to 
2003. The amount of scientific papers grows by seven to nine percent 
evety year (compounded), which equates to a doubling of scientific 
output roughly every ten years., 

While it is true that many people know how to search for information 
on the Internet, it is not clear that they have gtasped the more nuanced 
reasoning skills necessary to critically evaluate and synthesize what they 
find, especially when we consider the daunting amount of information 
they must process. 

Twenty-First Century Information Literacy Tools (TILT), a program 
of The People’s Science, has identified six core skills and sensibilities for 
interacting with and applying information in real-world contexts. These 
objectives outline the essential capacities that must be developed in order 
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to responsibly curate, evaluate, and transform an abundance of 
information into usable knowledge. 
TILT identifies the following core capacities of 21st century 


information literacy: 


e Maintain a dynamic disposition by accepting the progressive 
nature of information and remaining open to new evidence. 

e Consider the role of socio-cultural lenses in the interpretation of 
information and the proliferation of new ideas. 

e Cultivate comfort with competing evidence by acknowledging 
informed debate as a critical, nuanced step towards replication, 
refinement, and eventual consensus. 

e Evaluate source credibility for common access points in the 
information dissemination cycle. 

° Develop an informed orientation to ensure clarity of how specific 
evidence is situated in the broader landscape of relevant 
knowledge. 


With information output growing at an unprecedented rate, 
information literacy skills are increasingly important for all students 


throughout all subject areas. 


Digital Literacy 


As discussed above, technological savvy is becoming increasingly 
important. As tools and technologies continue to develop, students must 
learn to use a variety of new technologies. The majority of jobs are going 
to require upskilling, as we begin to integrate technological innovations 
across the majority of possible careers. It is important that students learn 
to be comfortable with existing technological tools, such as searching on 
the internet, word processing, spreadsheets, and social media 
applications, and that they feel comfortable learning new technologies. 
All these themes offer educators and students alike a way to make 
learning more relevant, grounded in the real world, motivating, and 
action-oriented. They also provide a foundation for interdisciplinary 


a: 
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thinking, as they are lenses that educators can mix and match to content 


areas and competencies. 


Systems Thinking 


Scientific disciplines as well as social systems are converging on the ideas 
of complex systems (see the following figure). This requires a paradigm 
shift from the mechanistic and reductionist model of twentieth century 
Western culture, toward a more balanced approach. Analysis continues to 
serve a critical purpose by isolating parameters, thereby allowing for their 
deep treatment and understanding, but it must be integrated with a 
holistic perspective through synthesis, such that each part can be 
considered as a whole, each whole as a part of a larger system, and the 
relationships between all of them explored. 


@ Chemisty Biology 


Mathematics 
Physics 


be Computer Science 
Simpie systems 


ee eel ee 
Philosophy Pa ] a 
Sociviogy 


Anthropology 


Economics Psychology 


Complex systems 


®©) 


Simple systems 


Systems Thinking as it relates to disciplines 
Source: Bar-Yam, Y. Dynamics of Complex Systems 


According to educational theorist and cognitive scientist Derek 
Cabrera, students should be encouraged to consider distinctions, systems, 
relationships, and perspectives (DSRP). 


e Distinctions: Develop increasingly sophisticated 


characterizations of ideas and objects. 
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e Systems: Deconstruct ideas and re-constructing new integrated 
concepts with a variety of part/whole interactions. 
e Relationships: See connections between things. 


e Perspectives: See things from different points of view. 


By considering the common properties of complex systems, learners 
can apply this approach to view more traditional disciplines from a 


modern, systems perspective. 
Design Thinking 


As we have seen, the twenty-first century challenges we now face are 
demanding a major rethinking and redesigning of many of our societal 
institutions from education, to agriculture and energy use, to product 
design and manufacturing, to economics and government. Almost every 
product and service needs to be redesigned in light of our increased use 
of information and communications technologies, global connectivity, 
enetgy and material ecological sustainability, longer lifespans, and 
increased well-being. Beyond products and services, a design thinking 
mindset is needed in the ways we approach our challenges. 

One way of crisply conceptualizing the design process is through four 
main principles: 


e The human tule: All design activity is ultimately social in 
nature. 

e The ambiguity rule: Design thinkers must preserve ambiguity. 

e The redesign rule: All design is redesign (mistakes are a natural 
part of the process of iterative improvement). 

e The tangibility rule: Making ideas tangible facilitates 


communication. 


Computational Literacy 


In discussions about how to best ptepare students for the 21st 
century, it is common to focus on the increasingly ubiquitous role of 


technology, and thus the need for students to learn computer science 
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skills. But experts have agreed that rather than any particular 
programming language or paradigm, what is most important is 
internalizing the types of thinking that is involved in mastery of 
Computer Science. According to the Royal Society, “Computational 
thinking is the process of recognizing aspects of computation in the 
world that surrounds us, and applying tools and techniques from 
Computer Science to understand and reason about both natural and 
attificial systems and processes.” In a recent review article, the following 


elements were defined as central to CT: 


e Abstractions and pattern generalizations (including models 
and simulations). 

e Systematic processing of information. 

e Symbol systems and representations. 

e Algorithmic notions of flow of control. 

e Structured problem decomposition (modularizing). 

e Iterative, recursive, and parallel thinking. 

e Conditional logic. 

e Efficiency and performance constraints. 


e Debugging and systematic error detection. 
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Appendix 2 
What is AIP 


In the preceding text, we gave a btief introduction to AI (sufficient to 
ground our discussion of the application of AI in education (AIED)). 
Here, we build on that primer, to give more details for readets who ate 
interested. 

With this in mind, where better to start than with some definitions, to 
provide some context for what follows. According to the Oxford English 
Dictionary, artificial intelligence is 


The capacity of computers or other machines to exhibit or 
simulate intelligent behavior, 


— The Oxford English Dictionary*38 


This definition confirms that artificial refers to computers or other 
machines, but it still begs the question what is meant by intelligent 
behavior, or indeed by intelligence. In fact, what is understood by 
intelligence and whether machines can be intelligent have long been 
controversial (for a seminal discussion, see the Chinese Room Argument, by 
the philosopher John Searle)” For Alan Turing (who is regarded by 
many as the father of both modern computing and artificial intelligence), 
we might consider a computer intelligent if it passes what he called the 
imitation game (and is now mostly known as the Turing test), in which a 
computer hidden from view imitates a human by responding to a series 
of conversational questions. 


I believe that in about fifty years’ time it will be possible to 
Program computers... to make them play the imitation 
game so well that an average interrogator will not have 
more than 70 percent chance of making the right 
identification after five minutes of questioning. 

—Alan Turing440 


SS ee 
“8 OED. (2018). “artificial intelligence, n.” OED Online. http://www.oed.com/view/ Entry/271625 
*? John R, Searle. (1 980). “Minds, brains, and programs.” Behavioral and Brain Sciences 3 (3): 417-424, 
“° A.M Turing. (1 950). “Computing machinery and intelligence.” Mind 59 (236): 433-460, 
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Introducing humans (the interrogator) to the discussion also suggests 


an alternative approach, comparing artificial with human intelligence: 


AI is the science of making machines do things that would 
require intelligence if done by men—Marvin Minsky**! 


Updated, this informs another dictionary’s expanded explanation. 


Artificial intelligence is 


[A] discipline concerned with the building of computer 
programs that perform tasks requiring intelligence when 
done by humans. Examples of tasks tackled within AI are: 
game playing, automated reasoning, machine learning, 
natural-language understanding, planning, speech 
understanding, and theorem proving. 


—A Dictionary of Computer Science? 


On reflection, and inevitably, there are probably as many definitions 
of artificial intelligence as there are books on the subject (and there are a 
good many, mostly technical, books). However, Russell and Norvig’s 
work—for many, the key AI textbook***—observes that most definitions 
of artificial intelligence fit into a matrix of four distinct approaches, 
depending on whether they prioritize thinking or behaving, and whether 
they take a human-like or rational perspective (as illustrated in the 
following table): (i) AI that aims to “hink like a human, by automating 
human cognitive abilities such as decision making and problem solving 
(as occurs in many AIED individualized tutoring systems (ITS)); (ii) AI 
that aims to act like a human, by automating human non-reflective abilities 
such as image and voice recognition (as is used in AIED DBTS and 
automatic essay feedback); (iii) AI that aims to think rationally, by means 
of computational models of perception and reasoning (as might be 


necessary in an AI teaching assistant); and (iv) AI that aims to act 


#1 Marvin Minsky (1968) quoted by Blay Whitby in (1996) Reflections on Artificial Intelligence. Intellect Books, 20. 
#2 “artificial intelligence.” In (2016). A Dictionary of Computer Science. Butterfield, A. and Ngondi, G.E. Oxford 
University Press. http://www.oxfordreference.com/view/10.1093/acref/9780199688975.001.0001 /acref- 
9780199688975-e-204. 

“8 Russell and Norvig, Artificial Intelligence. 
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rationally, by means of intelligent agents (as is used in many AIED 


[renee [A 


Thinking Thinking like a human. Thinking rationally. 
Automating human Computational models 
cognitive abilities of perception and reasoning. 
(e.g., decision making and 


simulations). 


problem solving). 


Acting like a human. Acting rationally. 
Automating human non- Designing intelligent agents 
reflective abilities (e.g., (artifacts with intelligent 
image and voice behaviors). 

recognition). 


Approaches to artificial intelligence. 
Source: CCR based on Russell and Norvig, Artificial Intelligence, p. 2. 


Returning to the use of the word artificial in artificial intelligence and 
Searle’s argument that a computer program can never be capable of 
independent thinking, perhaps we should stop altogether using the 
phrase artificial intelligence (intelligence exhibited by something that is 
not human), thus avoiding the hermeneutic controversy. Indeed, this 
effectively is argued by many writers, not least by Doug Engelbart“* and 
Vint Cerf” (two leading pioneers of modern computing) who refer 


446 


instead to what they call augmented intelligence’ (what others have called 


intelligence augmentation or intelligence amplification“). This phrasing dodges 
Seatle’s objections by retaining the human brain as the source of 
intelligence, and by positioning the computer and its programs as a 
sophisticated tool (or complex of tools) with which us humans might 


* Engelbart, D. C. (1962). “Augmenting human intellect: A conceptual framework.” Prepared for the Air 
Force Office of Scientific Research. Stanford Research Institute. 
https://www.dougengelbart.org/pubs/augment-3906.html 

+ Cerf, V.G. “Augmented Intelligence.” IEEE Internet Computing 17. https://doi.org/10.1109/MIC.2013.90 
“6 Pasquinelli, M. (2014). “Augmented intelligence, critical keywords for the digital humanities.” 
http://cdckeywords.leuphana.com/augmented_intelligence 

“7 https:// www.weforum.org/agenda/2017/01/forget-ai-real-revolution-ia 

“8 Ashby, W.R. (1956). An Introduction to Cybernetics. Chapman & Hall Ltd. 
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enhance or augment our intellectual capabilities. In this approach, 
computers are employed to do what humans find more difficult (such as 
finding patterns in huge amounts of data). In fact, some have argued that 
it is the combination of humans with AI, what others have called centaurs 
(half-human and half-Al)“’, that has the greatest potential, better than 
either humans or AI alone. 

Nonetheless, the debate contrasting augmented and artificial will 
inevitably run and run, with artificial intelligence winning at least on 
popular usage even if augmented intelligence is more accurate or useful. 
This is why, in most of the main text and hereafter, we take the ultimate 
pragmatic approach and refer almost exclusively to AI, leaving the reader 
to decide for themselves what the A in AI represents. 


A Short History of AI 


APs foundational event is widely considered to be a workshop held at 
Dartmouth College, a US Ivy League research university, in 1956.*° It 
was where what is thought to be the first AI program, the Logic Theorist, 
was presented and discussed. The Dartmouth workshop involved John 
McCarthy (who is usually credited with convincing his colleagues to 
name the new field artificial intelligence), together with others including 
Marvin Minsky, Allen Newell; and Herbert Simon, all of whom would go 
on to become leading AI researchers. 

Over the following decades, AI developed in fits and starts with 
periods of rapid progress interspaced with periods, known as AI winters, 
where confidence and funding all but evaporated. In its early years, AI 
research was centered on just a few labs, in particular, the Massachusetts 
Institute of Technology, Carnegie Mellon University, and Stanford 
University, but the scope of AI research quickly mushroomed. Here, to 
illustrate the breadth of early AI research, we will look briefly at four 


“ Case, N. (2018). “How to become a centaur.” Journal of Design and Science. 
https://doi.org/10.21428/61b2215c 
#0 Crevier, D. (1993). AI: The Tumultuous History of the Search for Artificial I ntelligence. Basic Books. 
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areas: rule-based systems, computers that play games, computer vision, 


and natural language processing. 


Rule-Based and Expert Systems 


Rule-based inference systems were, in fact, both an early focus and a 
long-lasting AI success. Especially memorable from the early days, a rule- 
based approach was implemented in ELIZA,*' which, among other 
applications, was programmed to impersonate a psychotherapist. ELIZA 
is self-evidently the original precursor of AIED dialogue-based tutoring 
systems (DBTS) such as AutoTutor. A brief extract of a typical ELIZA 
conversation, which was typed on a computer terminal, went like this: 


“My boyfriend made me come here.” 

ELIZA: “Your boyfriend made you come here.” 
“He says ’m depressed most of the time.” 

ELIZA: “T’m sorry to hear that you ate depressed.” 


In this way, ELIZA appeared somewhat responsive, perceptive and 
thus intelligent. However, it was only using rules, such as me—you 
sentence transformations (“My boyfriend” became “Your boyfriend”), 
inserting leading phrases (“I’m sorry to hear that”), and later returning to 
earlier randomly selected sentences as if it were making a novel logical 
connection. ELIZA is also notable for being one of the first AI tools to 
have been commercialized, despite it not having any psychological 
benefits.” 

Over time, rule-based systems developed into what became known as 
expert systems, “computer program|[s] that reason using knowledge to solve 


29 453 


complex problems,” œ“ with one early example being MYCIN which was 
designed to diagnose some infectious blood diseases** (many early AIED 


ITS are essentially expert systems). Like other expert systems, MYCIN 


#1 Weizenbaum, J. (1966). “ELIZA—a computer program for the study of natural language communication 
between man and machine.” Communications of the ACM 9 (1): 36-45. 

+2 Weizenbaum, J. (1976). Computer Power and Human Reason: From Judgment to Calculation. Freeman. 

* Feigenbaum, E.A. (1992). “Expert systems: Principles and practice.” In The Encyclopedia of Computer Science 
and Engineering. 

* Shortliffe, E.H., et al. (1975). “Computer-based consultations in clinical therapeutics: Explanation and rule 
acquisition capabilities of the MYCIN system.” Computers and Biomedical Research 8 (4): 303-320. 
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used a sequence of IF... THEN... and other conditional tules, in this case 
to infer a diagnosis from the result of blood tests and bacterial cultures. 


A simplified extract illustrates the process of logical inference. 


IF the bacterial culture is positive 

AND the entry is the gastrointestinal tract 
AND the abdomen is the locus of infection 
OR the pelvis is the locus of infection 


THEN ~ Enterobacteriaceae. 


Although expert systems typically contain many hundreds of IF... 
THEN... conditional rules, it is usually possible to follow the instantiated 
logic that leads to the outcomes—in other words, the system’s rules and 
decisions can be inspected. However, as the interactions between rules 
can quickly multiply, expert systems can also sometimes be challenging to 
understand and debug. Nonetheless, expert systems are relatively 
inexpensive to develop, are resilient to errors; and can be relatively 
flexible (if a new condition needs to be accounted for, a new rule can 
usually be added without compromising the existing syntax). For all these 
reasons, rule-based systems were quickly embraced by industry, although 
their use was usually limited to specialist applications (for example, 
encouraged by the fact that their only expert locomotive engineer was 
nearing retirement, General Electric developed an expert system that was 
capable of diagnosing 80% of electric locomotive repair problems).*”° 

In fact, developing expert systems for specialist applications 
highlighted a key problem, known as knowledge acquisition. How could 
the expert-system designers, who were not usually experts in the domain 
being represented, identify the knowledge that they needed to codify? In 
turn, this led to developments in both knowledge elicitation (methods for 
extracting knowledge from domain experts in a way that could logically 
be instantiated) and knowledge engineering (methods for instantiating the 
elicited knowledge in the expert systems’ conditional tules).*°° With these 


455 Crevier, AI, 198. 


“6 Cooke, N.J. (1994). “Varieties of knowledge elicitation techniques.” International Journal of Human-Computer 
Studies 41 (6): 801-849, 
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various methods now mature, expert systems are used in various contexts 
(such as manufacturing, agriculture, engineering, tax assessments, and 
loan-eligibility calculations), particularly in situations where human 
experts are mostly unavailable (due to location or cost). More recently, 
expett systems have been enhanced with new AI techniques, a 
development to which we will return later. 

Each of the other examples of early AI research that we will now 
discuss (playing games, computer vision, and natural language processing) 
saw early rapid advances. On the other hand, each is a good example of 
the “naïveté of early AI efforts’**’—the fact that early AI researchers and 
their funders anticipated greater advances than were actually realized. 


Computers Playing Games 


Some early AI researchers were interested in developing systems capable 
of playing games such as chess, backgammon, and checkers, because 
such games comprise only a few basic elements and are rule-bound yet 
have non-trivial solutions. Various approaches were investigated. One 
tactic involved trial-and-error search methods, in which the tree-like 
problem space of possible moves is searched to identify the optimal 
pathway to success (which in chess was always challenging because of the 
exponential number of possible moves at any one time). Another 
approach used evaluation methods, which more closely replicated how 
humans play, and involved defining criteria to evaluate the goodness of 
available moves and to choose the move with the best chance of success. 
Some systems also introduced statistical techniques to weight the choice 
of moves according to the outcomes of a large number of previous 
games, while others used pruning (removing obvious dead-ends) or 
heuristics (rules of thumb) to make problem-space searching more 
efficient. Many of these systems eventually were capable, thanks mainly 
to the ability of computers to quickly compute massive amounts of 
numerical data, of beating advanced human players, although it was not 
until 1997 that IBM’s chess playing Deep Blue managed to beat the 


47 Crevier, AI, 89. 
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reigning world chess champion, Boris Kasparov. It was not until twenty 
years later that Google’s DeepMind, using a very different AI, beat the 
world champion Go player, Lee Sedol. Nonetheless, many of the 
techniques pioneered by the early game researchers are still in use 
today—especially in the various gaming apps played by millions on their 
smartphones and tablets.. 


Computer Vision 


To develop computer vision, the ability of computets to recognize 
objects (to see) in the real world, the early researchers recognized that 
they needed to drastically simplify the problem. Rather than try to 
interpret the complexity of the sights that humans see (such as randomly 
shaped objects, textures, perspective, millions of colors, movement), 
they started with a highly simplified model: blocks micro worlds. The 
world was comprised of stationery geometric objects such as pyramids, 
cubes, and rectangular blocks. Various early attempts worked by first 
trying to identify corners, edges, and block faces, although many of these 
efforts failed. One key problem was that the systems did not understand 
occlusion (when one object partly hides another), an issue that was 
eventually solved by including various knowledge rules within the system 
(such as a line has two ends, and an atch has two non-touching upright 
blocks supporting a third block). Nonetheless, although many significant 
advances were made, overall progress in computer vision was slow. 


Natural Language Processing 


Early natural language processing research efforts were in machine 
translation, which had been funded since the early 1950s by the US 
government with the aim of automatically translating Cold-War Russian 
documents. However, the work was unsuccessful: the infamous 
retranslation (English to Russian to English) of “the spirit is willing but 
the flesh is weak” as “the vodka is good but the meat is rotten” illustrates 
the difficulties they were unable to address.‘ Funding was soon 


#8 Russell and Norvig, Artificial Intelligence, 21. 
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withdrawn. A much-simplified method was taken by SHRDLU,*” which 
adopted the approach of the blocks micro worlds, and their more modest 
ambitions, although SHRDLU’s blocks micto world existed only inside 
the computer’s memory. With SHRDLU, a person could engage in a 
simple, typed dialogue with the system, instructing a simulated robot arm 
to interact with (pick up and move) simulated block objects, by using 
commands expressed in everyday language (pick up the big red block), 
Like all early approaches to natural language processing, SHRDLU 
depended on the direct hand coding of conditional tules, an approach 
that—it was soon realized—could not cope with the full variation of 
natural languages. Although most languages have a stable underlying 
structure, all languages have innumerable surface differences and multiple 
ambiguities; such that, almost as soon as a language tule is encoded, an 
exception is identified, and another rule becomes necessary, in a 
seemingly endless cycle. 


The First AI Winter 


While all of these advances were promising, they all failed to live up to 
the researchers’ and funders’ hopes and expectations. Much of the early 
funding had come from the Defense Advanced Research Projects 
Agency (DARPA), which had given millions of dollars for AI research 
with almost no strings attached. However, following a report from the 
British Science Research Council*” that suggested most AI research was 
unlikely in the near future to produce anything useful, and the lack of AI 
technologies that might be applied usefully to DARPA’s work, the 
money was withdrawn—leading inevitably to the first AI winter, in which 
AI research slowed to a snail’s pace. 

The purpose of this brief look at the beginnings of AI research was to 
illustrate its ambitions, breadth, successes, and challenges, all of which 


had implications for the application of AI in education. In this appendix, 


* Winograd, T. (1980). “What does it mean to understand language?” Cognitive Science 4 (3): 209-241. 
*® Lighthill, J. (1973). “Lighthill report: Artificial intelligence: A paper symposium.” Science Research Council. 
http://www.math.snu.ac.kt/ ~hichoi/infomath/Atticles/Lighthill Report.pdf 
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we will now leap forward to the most recent decade,*” in which, thanks 
to three key developments—the advent of faster computer processors, 
the availability of large amounts of big data, and advances in 


computational approaches—AI has entered a period of renaissance. 


AI Today 


As we noted at the beginning of this appendix, AI has become an often 
hidden but integral, pervasive, and inescapable part of our daily lives. In 
fact, paradoxically, the more it is integrated into our lives, the less we 
tend to think of it as AI. 


A lot of cutting edge AI has filtered into general 
applications, often without being called AI because once 
something becomes useful enough and common enough 
it’s not labeled AI anymore.4& 

Instead, AI is often known as an advanced computer program (such 
as email spam filtering), a personal ant (such as Cortana), a 
recommendation system (such as in Netflix),“® or perhaps a language- 
learning app (such as Duolingo).*® Having said that, recent voice- 
activated smart speakers, such as Google Home“ and Amazon Echo,*® 
have made AI more visible in our living rooms. 

In fact, many recent developments in AI, due mainly to the three key 
developments noted earlier (faster computer processors, large amounts 
of big data, and new computational approaches), have been both 
groundbreaking and in many ways transformative. In fact, this appendix 


“*! Readers wishing to learn more about the history of AI might enjoy Daniel Crevier’s AI: The Tumultuons 
History of the Search for Artificial Intelligence, or the first chapter of Russell and Norvig’s comprehensive Artificial 
Intelligence. 

462 http://edition.cnn.com/2006/TECH/science/07/24/ai.bostrom index.html (Professor Nick Bostrom, 
director of the Future of Humanity Institute, University of Oxford). 

* E.g., https://www.mailwasher.net which uses Bayesian techniques to learn which emails are spam and 
which are not. 

464 https://www.microsoft.com/en-us/cortana 

* https://help.netflix.com/en/node/9898 

*° https://www.duolingo.com 

ast https://store.google.com/gb/product/google_home 

** https://www.amazon.com/b/?ie=UTF8&node=9818047011 
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is already necessarily out of date, with new AI techniques, tools, and 
products being launched all the time. 

AI techniques such as machine learning (supervised, unsupervised, 
and reinforcement learning), neural networks (including deep learning), 
and evolutionary algorithms have all been used in applications as diverse 
as autonomous vehicles, online shopping, auto-journalism, online dating, 
image manipulation,“ stocks and shares dealing, and legal and financial 
services. We will look in some detail at these core AI techniques. But 
first, to give a context, let us consider some recent AI applications. 


Face Recognition 


Automatic face recognition is one atea that has quite recently made 
qualitative leaps forward while simultaneously becoming almost invisible. 
It is the technology used in smartphone cameras to ensute that faces ate 
always in sharp focus and at e-passport gates to identify travelers before 
allowing them to enter a country. Earlier, we described the beginnings of 
computer vision, and the need to simplify the problem by focusing on 
blocks micro worlds. Following that work, there was slow progress, but 
neat-human-level computer vision remained far out of reach; at least 
until 2012, when Google applied a quite different computational 
approach. Rather than trying to program computer vision, Google 
tesearchers presented a brain-inspired AI neural network, made up of 
16,000 computer processors, with 10 million randomly selected video 
thumbnails from YouTube.*” By using deep-learning techniques, and 
despite not being told how to recognize anything in particular, this 
machine learning system soon learned how to detect human faces in 
photographs. Two years later, Facebook introduced a nine-layer deep AI 
neural network, involving more than 120 million parameters, to identify 
(not just detect) faces in timeline photographs.‘ It was trained on a 


“© https://www.bbc.co.uk/news/av/’ technology-45361794/ how-artificial-intelligence-can-edit-your-pictures 
#1 https://www.nytimes.com/201 2/06/26/technology/in-a-big-network-of-computers-evidence-of-machine- 
learning. html?_r= T 

#1 Facebook introduced a nine-layer deep AI neural network, involving more than 120 million parameters, to 


identify (not just detect) faces in timeline photographs. It was trained on a dataset of four million images. 
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dataset of four million images of faces that had previously been labeled 
by humans (the Facebook users, who had been happily labeling their 
friends in uploaded photographs over several yeats), and was able to 
achieve accuracy in excess of 97 percent, which almost matches human- 
level performance. However, although impressive, these examples also 
highlight a key difference between AI and human intelligence: a human 
doesn’t need to see ten, or even four, million faces before it can easily 
recognize a family member, a friend, or a celebrity (one of the ways in 
which humans still currently out-perform AI, although continual 
advances suggest this will not always be the case). 


Autonomous Vehicles 


Another area that has seen much AI development in recent years is 
autonomous vehicles, with neural networks being used to enable cars, 
trucks, and taxis to drive without human intervention. The avowed aim 
of the developers is to drastically reduce the number of road accidents, 
injuries, and fatalities on our roads, and to manage traffic jams out of 
existence. Autonomous vehicles depend on a complex rig of cameras, 
sensors, and communication systems, together with massive computing 
power, to enable them to see, hear, feel, think, and make decisions like a 
good human driver. As the vehicle drives along, its sensors detect the 
toad’s edges and markings, road signs and traffic lights, other vehicles 
including bicycles, other potential obstacles, and pedestrians (including 
those who run across the road without looking). Simultaneously, a neural 
network-driven intelligent agent controls the cat’s steering, acceleration, 
and braking. In fact, despite all the necessary AI, the success of 
autonomous vehicles (which ate not likely to be widely available any time 
soon) still depends on humans—the thousands of people, the wizards 
behind the curtain,*” whose job it is to manually label individual frames 
from thousands of hours of video footage captured by cars driving 


around the cities. In other words, the autonomous vehicle is an area of 


#2 https://www.ft.com/content/36933cfc-620c-11e7-91 a7-502£7ee26895 and 
https://www.bbc.co.uk/news/technology-46055595 
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AI research that still, and probably will for some time yet, depend on 


humans. 


Auto-Journalism 


A probably less well-known use of AI is in journalism. News 
organizations around the world are developing AI technologies to 
support their news gathering and news reporting. For example, AI agents 
continually monitor global news outlets and use semantic analysis to 
automatically extract key information, which is made available to 
journalists. to write their stories.*” There are even some AI technologies 
that go one step further and automatically write the stories themselves.“ 
They take the curated information, match it to a template, and create a 
story that can be published across multiple platforms. The output might 
be formulaic, but this approach has been very successful at writing 
accurate teasets that direct readers to the in-depth writing of the human 
journalists." Other uses of AI in journalism include moderating forum 
comments (using sentiment analysis to automatically detect aggressive or 
otherwise inappropriate postings) ,*”° data visualizations (in which the AI 
automatically determines the best way to display a particular set of 
data), chatbots designed to respond to user questions,*” and detecting 
fake news.*” But as well as detecting fake news, AI is also being used to 
create fake news" (if it can write real stories, it can write fake stories) 
and fake media (using an AI technology known as Deepfakes one can 
superimpose celebrity faces hyper-realistically on to actors in porn 
videos).*' “Think also of the potential abuse, by individuals or state 


* E.g., http://bbcnewslabs.co.uk/projects/juicer 

UMES https://narrativescience.com/Products/Our-Products/Quill 

*° E.g., https://www.washingtonpost.com/ pt/wp/2016/08/05/the-washington-post-experiments-with- 
automated-storytelling-to-help-power-2016-rio-olympics-coverage/?utm_tetm=.e22fladbfd5d 

“6 E.g., https://www.perspectiveapi.com 

#7 E.g., https://www.graphiq.com 

“8 E.g., https://www.theguardian.com/ help/insideguardian/2016/nov/07/introducing-the-guardian-chatbot 
 E.g., http://adverifai.com 

“0 E g., https:// www.technologyreview.com/s/610635/fake-news-20-personalized-optimized-and-even- 
harder-to-stop 

set https://www.unilad.co.uk/featured/the-real-reason-pornhub-has-banned-deepfakes 
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actors bent on spreading misinformation. Deepfakes could put words 


and expressions on to the face and mouth of a politician and influence 


. 482 
elections.” 


With AI being applied in increasing numbers of contexts, too many to 
cover fully in this appendix, we will conclude with three brief examples: 
legal services, weather forecasting, and medical diagnosis. 


AI Legal Services 


AI e-Discovery tools are being used to help lawyers process the huge 
amounts of documentation that need to be reviewed as potential 
evidence in civil or criminal legal cases, which can be a tedious and time- 
consuming task.’ One approach involves a machine-learning analysis of 
a sample of documents that have been reviewed and labeled by an expert. 
The outcomes enable the AI to then identify which of the remaining 
documents need to be prioritized for in-depth review. Similarly, AI tools 
have been developed to research relevant case law and statutes, to 
perform legal due diligence in mergers and acquisitions, and to undertake 


contract review as well as contract writing. 
AI Weather Forecasting 


In weather forecasting, machine learning has been shown to be more 
accurate at predicting weather than traditional simulation-based 
forecasting, Meteorologists have long tracked weather data that they 
enter into complex knowledge-based simulations to make forecasts. 
However, AI forecasting mines vast amounts of historical and immediate 
weather data, billions of data records from more than 1,000 weather 
satellites and 250,000 weather stations worldwide, with one company 
claiming to use more than 100 terabytes of data every day.“ These AI 


weather-forecasting systems use neural networks and deep learning to 


482 https://www.ft.com/content/8e63b372-8f1 9-11e8-b639-7680cedcc421 

485 hups://talkingtech cliffordchance.com/en/ emerging technologies /attificial-intelligence /ai-and-the-future- 
for-legal-services.html 

+8 McGovern, A., et al. (2017). “Using artificial intelligence to improve real-time decision-making for high- 
impact weather.” Bulletin of the American Meteorological Society 98. https://doi.org/10.1175/BAMS-D-16-0123.1 
#5 hetp://www.theweathercompany.com/ DeepThunder 
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identify data patterns (rather than to feed into simulations) in order to 


. . a. 486 
make data-based predictions about future weather conditions. 


AI Medical Diagnosis 


Our final brief example is the use of AI in medical diagnoses. For 
example, AI techniques are being used by radiologists to mote quickly 
identify anomalies in medical images while making fewer mistakes.*” One 
system looks for irregularities in X-ray images and, depending on what it 
finds, assigns it a priority. If it finds nodules on an image of a pair of 
lungs, it assigns a high-priority status and sends it to a pulmonary 
radiologist for further checks. Another system looks at retinal scans to 
detect diabetic eye disease, and has shown itself to be slightly more 
accurate than human ophthalmologists.“ Another recent innovation is 
online Al-driven general medical diagnoses, based on individuals 
entering descriptions of their symptoms and answering expert system-like 
questions, to enable the AI to come up with a diagnosis—although it still 
is unclear whether this type of diagnosis is as accurate as that of a teal 
doctor (only time will tell). 


AI Techniques 


While it is relatively straightforward to understand what the applications 
of AI outlined in the previous section are doing, understanding how they 
ate doing it can require some highly technical knowledge—exacerbated 
by the fact that any one AI application might draw on several different 
AI techniques. This is one reason why many people involved in AI have 
advanced degrees in mathematics or physics, although that is changing 
too. AI is increasingly being offered as a service: for example, Amazon’s 
Machine Learning on AWS,” Google’s TensorFlow,*”' IBM’s Watson,’ 


#% https://www.techemergence.com/ ai-for-weather-forecasting 

#7 Hosny, A., et al. (2018). “Artificial intelligence in radiology.” Nature Reviews Cancer 18. 
https://doi.org/10.1 038/s41568-018-0016-5 

so https://ai.googleblog.com/2016/1 1/deep-learning-for-detection-of-diabetic.html, and 
https://www.google.co.uk/about/stories/seeingpotential 

48 https:// www.babylonhealth.com 
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and Microsoft’s Azure,” Nonetheless, because some AI techniques have 
already been repeatedly mentioned, and because they play important roles 
in AIED and so will be mentioned again, some key and closely 
interlinked AI techniques and terminologies will next be inttoduced,*% 

Before moving onto our discussion of key AI techniques and 
terminology, we will first deal with two matters frequently raised 
whenever AI is the topic of conversation: so-called artificial general 
intelligence and the singularity. 


Artificial General Intelligence and the Singularity 


All the examples of AI mentioned so fat ate domain-specific or narrow. 
That is to say, the domain in which they operate is tightly constrained 
and very limited, and the AI, despite its sophistication, cannot be directly 
applied to any other domain. For example, the AI used to win at Go is 
incapable of playing a game of chess, the AI used to predict the weather 
is incapable of predicting movements in the stock market, and the AI 
used to drive a car is incapable of diagnosing a tumor. These applications 
of AI are all examples of what is sometimes referred to as artificial narrow 
intelligence, in contrast to so-called artificial general intelligence—AI that, like 
human intelligence, can be used in any circumstances. Artificial general 
intelligence has been defined as “AI systems that possess a reasonable 
degree of self-understanding and autonomous self-control, and have the 
ability to solve a variety of complex problems in a variety of contexts, 
and to learn to solve new problems that they didn’t know about at the 
time of their creation.” However, with some notable exceptions*”’ 
artificial general intelligence is rarely the focus of AI research and, 


contrary to some suggestions in the media, does not yet exist. In fact, its 


-e ee O 


491 https://www.tensorflow.org 

* https://www.ibm.com/watson 

#3 https://azure.microsoft.com 

* Readers wishing to learn more about AI techniques might be interested in Russell and Norvig, Artificial 
Intelligence; and Domingos, P. (2017). The Master Algorithm: How the Quest for the Ultimate Learning Machine Will 
Remake Our World. Penguin. 

#5 Goertzel, B. and Pennachin, C. (eds.). (2007). Artificial General Intelligence, Cognitive Technologies. Springer. 
*° E.g., https://opencog.org 
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complexity and the challenges that it brings should not be 
underestimated, such that artificial general intelligence is unlikely to exist 
in any meaningful sense for many years yet. 

The singularity is usually understood as the time in the future when AI 
is projected to be more intelligent than humans (interestingly, the word is 
taken from mathematics, where it refers to a property that cannot be 
described exactly because the equations no longer make sense). Already, 
algorithms are being developed that are capable of redesigning and 


#7 This, it is argued will soon lead to a spiral of 


improving themselves. 
rapidly increasing AI intelligence, “an intellectual runaway... faster than 
any technical revolution seen so far,’ until we attive at AI supet- 
intelligence,” AI that can conceive of ideas that no human being has 
ever imagined. This will be the singularity, the “end of the human era”! 
(which, fortunately, is not necessarily the same as the end of the human 
race). 

Unsurprisingly, while some welcome the singularity, believing that the 
supet-intelligent AI will innovate advanced technologies that will help 
address the world’s problems, others worty that it will not share our 
human goals and values, and that it will inevitably escape our control and 
possibly move against us.” In other words, for some, artificial general 
intelligence is necessarily a Pandora’s box (once open it can never be 
closed) with unknown and potentially catastrophic (the end of life as we 
know it) consequences. Nonetheless, just as artificial general intelligence 
remains an ambition, and despite recent rapid developments in AI more 
generally, the singularity is also unlikely to occur for many years yet. Even 
for the leading advocates of artificial general intelligence, the singularity 


#7 Kurzweil, R. (2006). The Singularity Is Near: When Humans Transcend Biology. Duckworth. 

#8 Vinge, V. (1993). “Vernor Vinge on the singularity.” Presented at VISION-21 Symposium Sponsored by 
NASA Lewis Research Center and the Ohio Aerospace Institute.” 

#2 Bostrom, N. (2016). S. uperintelligence: Paths, Dangers, Strategies. Oxford University Press. 

* E.g., Hawking. S., et al. (201 4). “Transcendence looks at the implications of artificial intelligence—but are 
we taking AI seriously enough?” The Independent. http://www.independent.co.uk/news/science/stephen- 
hawking-transcendence-looks-at-the-implications-of-artificial-intelligence--but-are-we-taking-ai-seriously- 
enough-9313474.html 
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appears to be due to arrive at some ever-receding future date, usually 


thirty or more years from the time that they are writing.” 


AI Techniques and Terminology 
Algorithms 


A process or set of rules to be followed in calculations or other 
problem-solving operations, especially by a computer, 
—The Oxford English Dictionary50 

As we mentioned briefly in the main text, algorithms are at the core of 
AI, such that the history of AI might be thought of as the history of the 
development of increasingly sophisticated and increasingly efficient (or 
elegant) algorithms. As we have seen, probably the most famous 
algorithm of recent times is PageRank, developed in 1996 by the 
founders of Google while they were students at Stanford University (and 
apparently named after Larry Page rather than web pages). It ranks the 
telative importance of a website, by counting the number of external 
links to the website’s Pages, to determine where the website appears in a 
Google search. 

In fact, all computer programs are algorithms. They comprise 
hundreds if not thousands of lines of code, representing sets of 
mathematical instructions that the computer follows in order to solve 
problems—compute a numerical calculation, grammar-check an essay, 
Process an image, or explain patterns that we see in nature.” All that 
makes AI algorithms distinct from other computer programs is that they 
involve some specific approaches and, as we have noted, they are applied 
to areas we might think of as essentially human—such as visual 
perception, speech recognition, decision-making, and learning. 


ee 


Müller, V.C. and Bostrom, N. (2016). “Future progress in artificial intelligence: A survey of expert 
opinion.” In Fundamental Issues of Artificial Intelligence, Springer, 553—570. 
http://link.springer.com/chapter/10.1007/978-3-319-26485-1 33 

i https://en.oxforddictionaries.com/definition/algorithm 
Turing, A. (1952). “The chemical basis of morphogenesis.” 
(641): 37-72. 
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In the following figure, we show a brief extract from an AI classifier 
algorithm (complete algorithms can run to thousands of lines), which 
itself calls a second algorithm called KNeighborsClassifier. 


lit iris data in train and test data 
; pee permutation, to split the data randomly 
. random. seed(@) 
— dice: = np.random.permutation(lenCiris_X)) 
>>> iris_X_train = iris_X[indices[:-10]] 
>>> iris_y_train = iris_y[indices[:-10]] 
>>> iris_X_test iris_X[indices[-10:]] 
>>> iris_y_test = iris_y[indices[-10:]] es 
>>> # Create and fit a nearest-neighbor classifier ae 
>>> from sklearn.neighbors import KNeighborsClassifier 
>>> knn = KNeighborsClassifier() 
>>> knn.fitCiris_X_train, iris_y_train) 
KNeighborsClassifierCalgorithm='auto', leaf_size=30, 
ic='minkowski' 
metric_params=None, n_jobs=1, n_neighbors=5, p=2, 
weights='uniform') 
>>> knn.predictCiris_X_test) 
array([1, 2, 1, @, 0, 0, 2, 1, 2, 01) 
>>> iris_y_test 
array([{1, 1, 1, 0, @, 0, 2, 1, 2, 01) 
A brief extract from an Al algorithm that in turn calls 
a k-nearest neighbor algorithm. 


Bayesian Networks 


Bayesian networks ate a type of graphical model employed by some AI 
applications. They represent the probability by which specified aspects of 
the world are related (their dependencies), and enable computational 
tasks such as prediction and diagnostics. A Bayes network graph (Bayes 
net) comprises various lines (known as edges) that connect nodes, on 
the nodes representing variables and the edges a 
interdependencies between those variables, as seen in the following 
figure. In a typical Bayes net, an edge from one node to another suggests 
that the first node causes the second, with a specific dependency 
probability. | 

To give a very simple example, an AI system might be required to 
predict (calculate the probability of) the flavor of ice cream that a 
customer might buy depending on the weather and temperature of the 
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day. Using a Bayes net approach, the nodes represent whether it is sunny, 
whether it is hot, and choices of ice cream flavor made by previous 
customers (all of which together constitute the known data) and what 
ice-cream flavor will be chosen today (an uncertain outcome) The Bayes 
net computation begins with probabilities given in each node that have 
been detived from training data (comprising records of weather, 
temperature and previous customers’ choice of ice-cream flavors) to 
derive the probabilities of various outcomes (the ice-cream flavors that 
will be chosen by customers in a combination of weather and 
temperature circumstances). 


Hot 
Sunny) rf 
T 08 02 
F 03 07 

Choice 


Sunny, Hot | cis 
iY (ee O 


T iF Os Os 
Fi T O06 04 
FiFf ios | os 


A simple Bayes net with conditional tables, predicting the ratio of chocolate 
and strawberry ice cream that will be sold, based on the weather, 
temperature, and sales of ice cream on previous days. 


The type of calculation that a Bayes net prediction involves, based on 
our ice-cream prediction, looks something like this (which is simplified 
and shown just to acknowledge their complexity): 

P(S=T,C =T) 
=T) 

This reads as: the probability of a chocolate ice cream being bought if 
it is sunny equals the probability of it being sunny, and a chocolate ice 
cream being bought divided by the probability of it being sunny! 
Fortunately, while it is important for AI engineers (but not so important 


(S=T)= 


Artificial Intelligence in Education 213 


for us) to understand exactly how this equation works, so they might 
decide the most appropriate approach in the particular circumstances— 
this is the type of computation that is provided by the AI as a service 
platforms mentioned earlier. 

In fact, typical AI Bayesian networks might comprise tens (or 
hundreds) of variables (nodes) with intricate interdependencies (edges), 
making the equations and their solutions increasingly complex. However, 
the Bayesian computational approach makes it possible to infer precise 
probabilities in uncertain environments in order to inform predictions 
that have practical uses (to continue with the example, to help the ice- 
cream seller decide how much of each ice-cream flavor to make). In fact, 
the Bayesian network approach is common in many expert systems, and 
has been used for applications as wide ranging as house price modeling, 
petrochemical exploration, tumor classification, victim identification ... 
the list is endless. 


Statistical Approaches to Natural Language Processing 


The ultimate goal of natural language processing (NLP) is to understand 
and generate written and spoken language as well as humans do (which 
chatbots like Siri, Cortana, and Echo demonstrate appears now to be 
increasingly possible). As we have noted, early linguistic-based NLP 
attempts were unsuccessful because of the huge variability in natural 
languages, which (if only because of the scale of the problem) proved 
impossible to encode effectively in conditional rules. Starting in the 
1980s, NLP researchers began to adopt an alternative approach, using 
statistical models. As with Bayesian nets, the NLP statistical models 
quickly become complex. Here, we will only introduce some examples of 
the multiple approaches taken in NLP. 

Statistical NLP begins with inferring properties of linguistic 
components from large corpuses of text (drawn from, for example, the 
Encyclopaedia Britannica, the New York Times, the Congressional Record). Each 
of the five basic types of linguistic components (phonology, morphology, 
Syntax, semantics, and pragmatics) needs to be addressed. One NLP 
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statistical technique is known as density estimation. The likelihood of a 
sentence being recognized correctly is evaluated based on its occurrence 
in the text corpus (and the context in which it appears). For example, 
“give peas a chance” is less likely to occur in the corpus than the similar 
sounding “give peace a chance,” so (assuming that the context is 
international politics) “peace” is mote likely to be the correct 
interpretation (on the other hand, if the context is young children’s 
dinner time ...). 

Inevitably, many sentences (especially longer sentences) do not 
actually appear in the corpuses of text. In these cases, the outcome can 
be refined using a second statistical model, such as one that assigns a 
probability to a sentence by computing the probability of each word in 
that sentence. The process goes like this: count the number of times 
“give peace a chance” appeats in the corpus, count the number of times 
“give peace a” appears in the corpus, divide these two numbers to 
compute the probability of “chance,” repeat for all the words in the 
sentence, and finally multiply the probability of each word to get the 
probability of the complete sentence! Other NLP statistical approaches 
involve segmenting words into prefixes, suffixes, and roots (un + friend 
+ ly), identifying parts of speech (noun, verb, and adjective), creating a 
profile for each word based on how often it appears next to another 
word (for example, “happy child” is more likely than “triangular child”), 
and identifying semantically similar nouns (for example, “boy” is mote 
similar to “woman” than it is to “car”’). 

Using these statistical techniques, NLP is designed for a range of 
applications, such as text summarization, information retrieval, question 
answering, and machine translation, and has the ultimate aim of dynamic 
human-to-computer conversation! We have already mentioned digital 
companion chatbots such as Siri, Google Home, Cortana and Echo; 
while one of the leading examples of machine translation is Google 
Translate, which has the capability to instantly translate speech and typed 
text (and photographs of typed text!) between more than a hundred 
languages. What is also particularly interesting about Google’s approach 
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is that the translations constantly improve over time: the more that it is 
used, the better that it gets.” In other words, statistical NLP is also an 
example of machine learning, to which we turn next. 


Machine Learning 


Programming computers to learn from experience should eventually 
eliminate the need for much ... detailed programming effort.503 


—A. L. Samuel 


As we saw earlier, when we introduced rule-based and expert systems, 
much AI (like much standard computer programming) involves writing 
in advance the steps that the computer will take to complete a task, the 
tules that will be followed exactly. Machine learning, on the other hand, 
is about getting computers to act without being given every step in 
advance. Instead of the algorithms being programmed exactly what to do, 
broadly speaking they have the ability to am what to do. This is not to 
suggest that machine learning does not require large amounts of 
programming, because it does. But rather that, instead of direct 
commands leading to direct outputs, machine learning involves large 
amounts of input data to predict novel outcomes. 

Machine-learning algorithms analyze the data to identify patterns and 
build a model, which is then used to predict future values. By identifying 
patterns in photographs of named people, it predicts who is shown in 
other photographs; by identifying patterns in medical symptoms, it 
predicts a specific diagnosis; and by identifying patterns in a student’s 
interactions in an ITS, it predicts learning pathways that are most likely to 
lead to success. In other words, machine learning may be considered a 
three-step process—(i) analyze data, (ii) build a model, (iii) undertake an 
action—that is continuously iterated. The outcomes of the action 


5% While most translation approaches require corpora of already translated matching texts, Facebook AI 
researchers have been exploring a method that only requires un-matched texts (i.e., different texts) in the two 
languages. https://www.forbes.com/sites /samshead/2018 /08/31/ facebook-develops-new-ai-technique-for- 
language-translation/ - 7b435£802£71 

* Samuel, A.L. (1959). “Some Studies in Machine Learning Using the Game of Checkers.” IBM Journal of 
Research and Development 3 (3): 210-229. 


Pi 


216 Artificial Intelligence in Education 


generate new data, which in turn amends the model, which in turn causes 
a new action. It is in this sense that the machine is learning. 

Many of the applications that we have looked at (including natural 
language processing, self-driving cars, and digital companions, as well as 
the Google DeepMind AlphaGo program that beat the world’s number 
one player of Go)™* have all been made possible thanks to machine 
learning. In fact, machine learning is so widespread today that, for some 
commentators when they speak of AI they mean machine learning and 
vice versa—whereas machine learning is more properly a sub-field of AI. 
What is true, however, is that the renaissance and exponential growth of 
AI over the last decade has come about because of significant advances 
in machine learning (based on faster computer processors, the availability 
of large amounts of big data, and new computational approaches).°” 
With that in mind, we will now move on to consider in more detail the 
specific machine learning approaches that we introduced in the main text, 
beginning with the three overarching categories: supervised, 


unsupervised, and reinforcement learning. 
Supervised Learning 


In supervised learning the agent observes some example input-output 
pairs and learns a function that maps from input to output 598 
—Russell and Norvig 
Most practical machine learning involves supervised learning. The AI is 
first provided large amounts of data for which the output is already 
known—in other words, data that has already been labeled. For example, 
the AI might be given many thousands of photographs of streets in 
which the numerous visible objects (bicycles, road signs, pedestrians) 
have already been identified and labeled by humans, or many thousands 


of snippets of speech that have been transcribed by humans, or many 


5% https://www.theguardian.com/technology/2016/mar/15/googles-alphago-seals-4-1 -victory-over- 
grandmaster-lee-sedol 

5 Interestingly, the origins of machine learning can be traced back to at least 1959, with the publication of 
“Some Studies in Machine Learning Using the Game of Checkers” cited earlier. 

5° Russell and Norvig, Artificial Intelligence, 708. 
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thousands of examples of symptoms with their diagnoses already 
determined by human doctors. The supervised-learning algorithm aims to 
identify the function that links the data to the labels, from which it builds 
a model that can be applied to new similar data. This is, broadly speaking, 
the approach used by Facebook to identify people in photographs we 
mentioned earlier, which used millions of photographs submitted and 
labeled by Facebook users to identify and label automatically the same 
people in new photographs. Returning to our other examples, a model 
built from the patterns detected in the labeled photographs of the streets 
can be used by autonomous vehicles to identify obstacles that they have 
to negotiate; a model built from the patterns detected in the labeled audio 
snippets can be used to automatically identify words in other recordings; 
while a model built from the patterns detected in the symptoms and 
diseases can be used to automatically diagnose patients who present 
similar symptoms. 

There are, in fact, two types of supervised learning, classification (when 
the output is a category, such as obstacle/not obstacle) and regression 
(when the output is a continuous variable, such as time or weight). There 
are also many supervised learning algorithms, with possibly the best 
known being k-nearest neighbors, linear regression, random forest, and 
support vector machines (the algorithms are named here so that, if you 
come across them again, they will be recognized for what they are). 

To give just one simplified example, we will look briefly at k-nearest 
neighbors. Imagine that you have some data that has been labeled with 
two categories (obstacle, not-obstacle). If this data is plotted in a 
scatterplot, each category of data points will be clustered together (in the 
scatterplot there will be one cluster of data points for obstacle and one 
cluster of data points for not-obstacle). If a new data point (representing 
a new object) is then plotted on the scatterplot, the k-nearest neighbor 
algorithm will decide to which cluster it belongs (in other words it will 
predict whether the new object is an obstacle or a not-obstacle), 
depending on the closest existing data points (the nearest neighbors). If 
the AI engineer decides to use a & of 5 (and there are many factors to 
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take into account), the algorithm will check the 5 nearest neighbors to 
make its decision. 


Unsupervised Learning” 


In unsupervised learning the agent learns patterns in the input even 
though no explicit feedback is Supplied.>'0 
—Russell and Norvig 

In unsupervised learning, the AI is provided with even larger amounts of 
data, but this time data that has not been categorized or classified, that is 
to say data that is not labeled. By analyzing this unlabeled data, 
unsupervised learning algorithms aim to uncover hidden patterns in the 
underlying structure of the data, clusters of data that can be used to 
classify new data (this is broadly the approach, mentioned earlier, used by 
Google to detect faces in photographs). Example applications of 
unsupervised learning include dividing online shoppers into groups, so 
that they might be served tightly targeted advertisements; identifying 
different letters and numbers from examples of handwriting; and 
distinguishing between legitimate and fraudulent financial transactions. 

There are, again, two types of unsupervised learning, clustering (for 
example, grouping handwritten letters by their characteristic shapes) and 
association (for example, identifying that people who watch one type of 
comedy also tend to watch a particular type of action movie). And, again, 
there are many unsupervised learning algorithms, the most common 
including k-means clustering, hierarchical clustering, ptincipal component 
analysis, and singular value decomposition. 

To give just one simplified example, k-means clustering automatically 
clusters the data into & clusters (if & is 3, the algorithm will cluster the data 
into 3 clusters; p, d and b handwritten letters). The algorithm involves 
several steps: (i) a potential center point for each cluster (known as a 


centroid) is selected randomly; (ii) the algorithm assigns each data point to 
=a 

A comprehensive list of the algorithms available on one of the leading AI as a service platforms, Microsoft 

Azure, is available at http://download.microsoft.com/download/A/6/1/A613E1 1E-8F9C-424A-B99D- 

65344785C288/microsoft-machine-learning-algorithm-cheat-sheet-v6.pdf 

510 Russell and Norvig, Artificial Intelligence, 708. 
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a centroid by calculating which is closest (which initially will result in 
some very uneven clusters); (iii) it calculates the mean of the distances 
between each data point and its centroid; (iv) it repositions the centroid 
to the position indicated by that mean; and (v) it reassigns the data points 
to centroids as before (which will result in some data points staying in the 
same cluster and others being reassigned to one of the other clusters). 
Steps (ii) to (v) are then repeated until no data points change clusters 


(which results in the most even clusters possible). 
Reinforcement Learning 


In reinforcement learning the agent karas fiom a series of 
reinforcements—rewards or punishments. 
—Russel and Norvig5!! 


In some senses, reinforcement learning is the most powerful of the 
machine learning categories. In both supervised and unsupervised 
learning, the model derived from the data although potentially powerful 
is fixed, and if the data changes the analysis has to be undertaken again 
(in other words, the algorithm is run once more). However, 
reinforcement learning involves continuously improving the model based 
on feedback—in other words, this is machine learning in the sense that 
the learning is ongoing. The AI is provided with some initial data from 
which it derives its model, which is evaluated, assessed as correct or 
incorrect, and rewarded or punished accordingly (to use a computer 
game metaphor, its score is increased or reduced). The AI uses this 
positive or negative reinforcement to update its model and then it tries 
again, thus developing iteratively (learning and evolving) over time. For 
example, if an autonomous car avoids a collision, the model that enabled 
it to do so is rewarded (reinforced), enhancing its ability to avoid 
collisions in the future; if a medical diagnosis leads to a patient’s health 
improving, the model is again reinforced, enabling the system to diagnose 
accurately future patients; and if Google AlphaGo makes a mistake that 
leads to it losing a game, its model is punished, improving its chances of 
not making the same mistake again. Finally, once again, there are many 
examples of reinforcement-learning algorithms, including Q-Learning, 


*" Russell and Norvig, Artificial Intelligence, 708. 
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State-Action-Reward-State-Action, and Deep Q Network—all of which, 
however, are too complex to describe simply. 


Category Characteristics Aim Example Algorithms 
Supervised Learns from Automatically k-nearest neighbors 
Learning labeled data. label new data. linear regression 


random forest 

support vector machines 
Automatically k-means clustering 
identify patterns hierarchical clustering 
(clusters) in the principal component 
data. analysis 

singular value 

decomposition 


Learns from 
unlabeled data. 


Unsupervised 
Learning 


Reinforcement Continually Continually Q-Learning 

Learning learns by improve the State-Action-Reward-State- 
rewards and outputs of the Action 
punishments. model. Deep Q Network 


The three overarching machine learning categories: supervised, 
unsupervised, and reinforcement learning. 


Artificial Neural Networks 


An artificial neural network is an AI algorithm based on the structure and 
functions of biological neural networks (such as animal brains), that 
might be applied in advanced supervised, unsupervised, or reinforcement 
learning. Our brains are made up of billions of individual neurons, each 
of which is connected (at synapses between axons and dendrites) to as 
many as a thousand other neurons, giving trillions of connections. 
Memory is thought to emerge from complex combinations of these 
connections across the brain, while learning is thought to involve the 
strengthening of those connections (a process known as Hebbian 
learning, often summarized as cells that fire together, wire togethe 5”? 
Although artificial neural networks have been trained to do some 
incredible things (such as driving a car without human intervention, or 


identifying faces in moving crowds of people, or beating the world’s best 


>? Lowel, S. and Singer, W. (1992). “Selection of intrinsic horizontal connections in the visual cortex by 
correlated neuronal activity.” Science 255 (5041): 209-12. 


Artificial Intelligence in Education 221 


Go players), they remain primitive in comparison to higher-order animal 
brains. Unlike, for example, the human brain’s billions of neurons, they 
usually involve only a few thousand neurons (in some exceptional cases, a 
few million); and, again unlike human brains, the neurons in artificial 
neural networks are arranged in logical layers, with each neuron in one 
layer being connected only to each neuron in the layers immediately 
before and after, as shown in the following figure. For these reasons, it is 
probably better to think of artificial neural networks as being inspired by 
biological neural networks, rather than being direct implementations 
(although, who knows what might be developed over the next few 
years?). 

As illustrated in the following figure, artificial neural networks each 
comprise three types of layers: an input layer (that takes stimuli from the 
environment, in the form of millions of data points, perhaps pixels from 
images), at least one, but often many more, hidden, intermediary layers 
(that together undertake the computation), and an output layer (that 
delivers the result). As mentioned, all the artificial neurons in one layer 
ate connected to each artificial neuron in the layers immediately before 
and after. In fact, each of these connections has a weighting. It is the sum 
of the weights received by one artificial neuron, whether that weight 
crosses some predefined threshold, that determines the weight of that 
artificial neuron’s outgoing connections (whether it is excited or 
inhibited, again in a process inspired by synapses in animal brains). 
During the machine learning process, it is these weightings that are 
adjusted in a process of reinforcement learning, and that allow the 
artificial neural network to subsequently compute outputs for new 
stimuli. Note that the figure on page 72 is vastly simplified for 
comprehension’s sake, as modern implementations have up to 50 or 
more layers and many different topologies. 

The hidden layers are key to the power of artificial neural networks, 
but they also bring an important problem. It isn’t possible (or at the very 
least it isn’t easy) to interrogate an artificial neural network to find out 


how it came up with its solution (for example, how did it identify a 
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particular person in a photograph?). In other words, artificial neural 
networks can lead to decision making for which the rationalization is 
hidden and unknowable, or un-inspectable, and possibly unjust, °° a 
critical issue that is the subject of much research.” 

To conclude this whirlwind tour of AI techniques and terminology, 
we will finish with three often-heard procedures: backpropagation, deep 
learning, and evolutionary learning. 


Backpropagation 


Backpropagation, short for backward propagation of errors, is an 
algorithm for the supervised learning of artificial neural networks. In our 
description of artificial networks, we desctibed a process where the 
outputs of one layer of neurons influence the neurons in the next layer, a 
process that moves only in a forward direction. However, some artificial 
neural networks also involve information moving in a backward 
direction. The actual outcome of the artificial neural network is 


compared with the desired output, which in turn influences the neurons 
in the hidden layers and their weightings, 


Deep Learning 


An extension of machine learning is known as deep learning, which 
involves artificial neural network algorithms comprising many hidden 
layers and a process of iterative clustering. For example, once a deep- 
learning algorithm determines that a picture contains a particular shape, it 
cycles again to find other shapes, and then cycles again to identify the 
connections between those shapes, iterating repeatedly until it has 


recognized what it is looking at (for example, a face). Deep learning is the 


headline approach used by AlphaGo. 


ee ee 
513 O’Neil, C. (2017). Weapons of Math Destruction. 


*4 Morcos, A.S., et al. “On the importance of single directions for generalization.” ArXiv.org, _ 
http://arxiy.org/abs/1 803.06959 
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Evolutionary Machine Learning 


An intriguing and cutting-edge field of research is evolutionary machine 
learning, an alternative approach to deep learning that uses a process 
inspired by Darwinian natural selection.” Evolutionary algorithms are 
the AI version of genetic algorithms,” which were introduced in 1960 by 
John Holland. While deep learning is focused on modeling what we 
already know, evolutionary machine learning is focused on creating 
solutions that do not yet exist. Instead of an AI engineer writing the final 
AI code, the evolutionary learning algorithm itself generates many pieces 
of random code, each of which it then evaluates for its fitness (Does the 
code do anything useful?). While code that is unsuccessful (is a poor fit) 
is abandoned, the code that is most successful (best fit) is then randomly 
mutated to produce many new pieces of code, all of which are again 
evaluated and subjected to survival of the fittest selection. This process is 
iterated many times, with the outcome being an Al-written new AI 
program. Evolutionary machine learning is currently the focus of much 


eatly research. Only time will tell what it will achieve. 


*8 Charles Darwin. (1869). On the Origin of Species by Means of Natural Selection: Or the Preservation of Favoured Races 
in the Struggle for Life. D. Appleton. 
316 https://en.wikipedia.org/wiki/Genetic_algorithm 
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About CCR 


Redesigning Education Standards 


The Center for Curriculum Redesign (CCR) is an international convening 
body and research center seeking to expand humanity’s potential and 
improve collective prosperity by redesigning K-12 education standards 
for the twenty-first century. In order to create a comprehensive set of 
frameworks, CCR brings together constituencies with diverse points of 
view—international organizations, jurisdictions, academic institutions, 
corporations, and nonprofit organizations including foundations—to 
consider and respond to the question: “What should students learn for 


the twenty-first century?” 


The Center’s Guiding Principles 


A sustainable humanity—one in which collective potential is expanded, 
and collective prosperity improved—is orchestrated out of multiple 
social, economic, and environmental factors. Key among them: a relevant 
education, based on meaningful curriculum, is critical to creating 
sustainability, balance, and wellbeing. 

While significant attention is being paid to teaching methods and 
pedagogy, the CCR argues that the what of K-12 education is at least as 
important as the bow, and brings a singular focus to the what. 

That twenty-first century what must take into account the accelerated 
pace of change we are experiencing, and shifts in societal and personal 
needs. Curriculum must be useful for the lives children will live and 
adapted accordingly. 

Our ability to contribute a meaningful WHAT requires openness to 
different perspectives. Therefore, CCR avoids dogma and emphasizes 
innovation and synthesis—multiple inputs applied and organized for 
optimum clarity and impact. 


We can—and will—shape the future we want. 
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Focus on the What 


Exponential changes in technology make specific predictions about the 
future all the more unreliable, but one thing is certain: we must prepare 
children to deal with greater complexity than ever before. The last major 
curriculum reform occurred in the late 1800s, also in a time of rapidly 
changing needs. Well into the twenty-first century, we can ill afford to 
depend on a nineteenth century curriculum. Indeed, we cannot expect 
our children to thrive unless we deeply examine, redesign and deliver a 
curriculum consistent with twenty-first century needs—one that is 
balanced and flexible. To thrive will mean to be adaptable, versatile and 
wise. 

In designing a curriculum framework around adaptability, versatility 


and wisdom we accomplish two main goals: 


e° Enhance the chances of an individuals personal and 
professional success and fulfillment. 

e Provide a common base of understanding and ability to 
participate in society, for a sustainable humanity. 


The Center’s Work 


The Center for Curriculum Redesign is not a program or intervention. 
The staff and CCR’s partners approach their work holistically, actively 
engaging with policymakers, standard setters, curriculum and assessment 
developers, school administrators, heads of schools, department heads, 
key teachers, EdTech experts and other thought leaders and influencers 
to develop a thorough understanding of the needs and challenges of all 
education stakeholders. This is essential to creating the vision of 
meaningful, relevant twenty-first century education, and to enabling 
practical implementation. 

The organization’s research, findings and recommendations ate 
actively disseminated through a wide variety of formats: CCR-sponsored 
conferences and seminars, active web presence and social media, 


consulting engagements and keynoting. 
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The following video serves to summarize our views, and can be 
shared freely: http://bit.ly/CCRintrovideo”’ About the Authors 
u u 
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